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PREFACE 


The International Forum on Participatory Evaluation was held in Delhj 
from March 1-5, 1988. It was the outcome of a collaborative 
arrangement among the International Participatory Research Network, 
the International Council for Adult Education, the Asian and South 
Pacific Bureau of Adult Education, and the Society for Participatory 
Research in Asia. Attended by 40 participants from 16 countries, the 
Forum proved to be an excellent event in sharing and analysing 


experiences, building linkages and inspiring future practice in different 
contexts and countries. 


It is always difficult to capture the spirit and the life of an event of 
this nature in words. Yet a record of that experience is necessary for the 
Participants, and more importantly for others as a vehicle for sharing 
the spirit and the insights of the Forum. This report attempts to 
capture, in a somewhat fragmented and perhaps inadequate manner, 
the depth of the discussions, the intensity of analysis and the spirit of 
camaraderie created and sustained during this Forum. 


In order to do justice to the spirit of the Forum, the report is divided 
into three parts. The first part describes the process—the process of 
coming together, creating a learning environment and working 
through issues, agendas and experiences. This is a record of the highs 
and lows, the excitement and the frustration of the process of coming 
together on this very important theme. 


The second part of the report attempts to encapsulate some.of the 
key issues debated, discussed and analysed during the Forum. These are 
the issues of Participatory Evaluationissues of theory, practice, steps, 
methods, methodology, the role of the facilitator, etc. They are 
recounted here with their diverse perspectives and positions, without 
necessarily attempting to provide a final answer to any of them. 


The third and final part reproduces the case studies which 
participants brought, shared and discussed during the Forum. The 
method of presentation of each person’s own experience, has not been 
altered, only minor editorial changes have been made. 


It is hoped that the totality of this record would provide some flavour 
of the experience that the Forum was, and that this record would then 
become a catalyst for similar reflections elsewhere. It could also be a 
basis for further refining our analysis and deepening our practice, as 
well as a starting point or a contribution towards strengthening our 
linkages across the world. 


THE PROCESS 


Tring! Tring! Tring! Tring. It is 2 O'clock in 
the morning and the phone rings. 


“Hello! This is Malina Demontis, calling 
you from the airport.” 


“Delhi Airport?” 


“Yes! Where is my visa? They will not let 
me in!” 


“Hello Malina, Rajesh Tandon here. How 
are you? Welcome to India, welcome to Delhi. 
Yes, we will get your visa across in half an 
hour. Please hold on. I know you must be tired 
after a long journey from Nicaragua. Yes, and I 
look forward to seeing you tomorrow. Bye!” 


Malina Demontis finally joined us at the 
Forum. 


The very process of getting these people 
together during the same week, at the same 
location, from different parts of the world was 
in itself a major achievement. In a world of 
shrinking boundaries and increasing regula- 
tions, it is becoming more and more difficult to 
get people together to share their experiences 
and to build linkages across regions and 
nations. One by one, they all came... from Mex- 
-ico, Chile, Mali, Ghana, Kenya, Swaziland, 

North America, England, Indonesia, Philip- 
- pines, Malaysia, Sri Lanka, Bangla Desh and 
different corners of India. They came and 
began to settle down at the place of stay which 
was also the place of the encounter. They came 
with their worries of return reservations, of 
whether they would be met at the airport or 
not, whether they would understand the lan- 
guage, whether it would be too hot or too cold, 
and what would the experience of the Forum 


be like. 


1. Building a learning environment 


The first process; therefore, was the process of 
building a learning environment. With all 
these anxieties, fears, doubts, hopes and expec- 
tations, women and men with diverse expe- 
riences, backgrounds, cultures, languages, 
nations and religions came together to work 
on the theme of Participatory Evaluation. In 
order to be able to share and learn from each 
other’s experiences, to deepen our analysis and 
reflection on the theory and practice of Parti- 
cipatory Evaluation, and to build ideas and 
plans for future follow-up, it was necessary to 
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create an environment of psychological safety, 
ease and relaxation for the participants. This 
was gradually done through informal 
exchanges and the recognition of each person 
as an individual, responding to their needs for 
creature comforts, setting forward the objec- 
tives of the Forum at the outset, creating an 
easy-going, comfortable pace of reflection 
through introductions of participants of their 
work and their experiences, and by re-designing 
the room in which we were sitting—making it 
more informal, face-to-face, direct and comfor- 
table. We also developed the agenda and the 
frame for analysis and reflection step by step, 
by continuously incorporating new ideas, 
points of view and perspectives into the main- 
stream of the discussions, by inviting persons 
from Latin America, Africa, Asia and Europe 
to share their initial reflections on participa- 
tory evaluation, which were then discussed in 
the plenary, and by identifying key issues of, 
concern and interest for further deliberation. 
Thus, a process of creating an appropriate and 
exciting learning environment was set in 
motion. 


Of course, the pressures of time, the difficul- 
ties of communicating in different languages, 
the-newness of the setting and the people, the 
hesitancy of expression, and anxiety over 
return reservations were very much a part of 
this initial phase. They had to be dealt with 
gracefully, with humility, with magnanimity 
and ease. Space had to be created for disagree- 
ments, for distractions, for rambling, story- 
telling, deviations from crisp, theme-focussed 


“statements. Opportunities had to be created to 


encourage people to say what was uppermost 
in their minds, despite the long time that was 
taken. The large and unwieldy plenary had to 
be maintained after lunch to give space to 
reactions and responses in order to set a frame 
for the issues which could then become the 
basis for deeper sharing and analysis, in the 
groups. The morning of March 1 and part of 
the afternoon went into this exercise. The time 
was now set to go deeper into sharing of 
experiences. 


2. Sharing of experiences 


_ How could the sharing of experiences be struc- 


tured? A large group of 40 may not be feasible. 
Could we have themes? Could we have levels 
of evaluation? Could we have types of exam- 


ples? There were many who had valuable, 
relevant and exciting experiences to share. 
How could we accommodate them all? We 
decided to form four groups on the basis of 
differences and similarities. We mixed people 
with diverse experiences at the base, at the 
organizational level, and at the level of pers- 
pective. We thought that this diversity will 
spark off exciting sharing, reflection and ana- 
lysis. From the late afternoon of March 1 till the 
entire day of March 2, we worked in these 
small groups. 


Each group had three or four key experien- 
ces to share. The experiences were related in 
detail; questions were asked, further details 
were sought, information was elicited. The 
narration and the presentations were then 
used to relate other experiences, to raise issues, 
to highlight theoretical and practical concerns. 
In some of the groups, the sharing of experien- 
ces was through the written case studies which 
were first read, then presented, then discussed. 
In some others, oral presentations were made 
with the help of flipcharts, drawings, illustra- 
tions. In yet some others, experiences were 
shared through enactment and role-play, in 
order to highlight peculiar dynamics in the 
experience of Participatory Evaluation. Some 
groups sang and danced together in order to 
narrate and describe their experiences. Some 
sang and danced together in order to show a 
spirit of camaraderie and solidarity. Over this 
period of a day and a half, the groups became 
living organisms with a life and direction of 
their own, with a sense of urgency and inten- 
sity in sharing, learning, describing, relating 
and at the same time theorizing, highlighting 
and evaluating. The experience of working in 
groups became the basis for relating deeply, 
intensely and personally to members of 
diverse cultures and experiences in the Forum. 
These groups became building blocks of the 
social organism that the Forum became. 


3. Representing issues 


Then came the time to relate to others what 
each small group had done. It came time to 
represent, highlight, refine and articulate 
issues discussed, as well as the experiences of 
each small group over the last day and a half. 
We came together on the morning of March 3 
as a plenary, and in that plenary, each group 
made its presentation. One group had four of 
its members describing the issues and emerg- 
ing themes. The second group brought two 
persons to present most of the discussions held 
in the group. A third group had every member 


of the group describe the iss.1e by becoming the 
issue, enacting the issue, narrating the issue, 
personalizing the issue and highlighting the 
issue. This was an experience which, at one 
level, visually and profoundly presented the 
issue and at another level highlighted the 
power of the method in expressing and relat- 
ing to the issue. With all the four presentations 
in, the exercise to identify common themes, 
visions and perspectives took place in the ple- 
nary. We went through a process of consensual 
validation, of recognizing that we were speak- 
ing the same language, and had shared similar 
experiences and common concerns. 


At this point we picked up a set of issues 
which we thought we could go deeper into. 
The small group exercise had so far helped 
identify issues through narration, expression 
and analysis of experiences. Now was the time 


.to describe, debate, analyse and deepen our 


understanding of the issues in order to move 
forward our practice of Participatory Evalua- 
tion. 


4. Analysing concepts 


In order to do this, we again worked in small 
groups from the afternoon of March 3. And we 
decided that we would reassemble on the 
morning of March 5 with presentations. Each 
group was now constituted on the basis of 
interest on the topic, so there was a group of 
five and another of 22. The group of 22 decided 
to meet at first as a whole group and then split 
into two sub-groups with similar but some- 
what varied focus. Each group worked on the 
afternoon of March 3, identifying, debating 
and analysing the issue. The group members 
expressed their opinions on the issue, related it 
to their own experience and theorized from 
that. The process of analysis culminated on the 
morning of March 5, when these groups pres- 
ented their-analysis of issues. Again, the pres- 
entations varied —some stood up and read out 
the statements they had prepared. Others used 
flipcharts to highlight nuances of the issues 
presented by several members of the groups. 
Some others enacted a role-play with different 
aspects of the issue being highlighted differ- 
ently; and some others sang. This step of ana- 
lysing the concerns and the issues helped in 
moving the reflection process further and in 
deepening our understanding of the metho- 
dology of Participatory Evaluation. 

5. Sharing colour 

In the midst of the Forum, the Festival of 
Colours, the day of Holi (March 4), was cele- 
brated. This is a day of colour, of joy, of celebra- 
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tion, of hugging and sharing. The participants 
in the Forum joined in this celebration, in 
particular participants from outside India. We 
spent the morning throwing colour, drinking 
bhang, singing, dancing, cajoling, pushing. We 
took ‘Holi’ dips in coloured water, some of us 
with a great deal of enthusiasm and energy, 
and some others with coy, shy and leave-me- 
alone ways. The day of Holi became the high- 
light of the Forum. It represented through the 
camaraderie, the togetherness, the solidarity of 
all people, all hamans—women and men from 
diverse cultures, languages, regions, religions 
and countries, sharing the essential humanity 
and the vision for a better, more humane and 
equitable world. It was an experience we all 
shared and would remember for a long time to 
come. 


6. Refining practice 


Following the small group presentations on 
the morning of March 5 we decided to then 
find ways to improve, refine and sharpen our 
practice within our own context, our own 
countries and regions. We began to look at the 
issues in relation to our practice, linking them 
to ourselves and our work. We sat in our coun- 
try or regional groups and found ways by 
which the experience of the Forum could be 
utilized in the future. We began some follow- 
up planning, identified some steps for sharing 
and networking in the future, in order to sup- 
port each other and enhance each other’s skills 
‘n carrying out the methodology of Participa- 
tory Evaluation. We then shared with all the 
participants the plans of the regions and the 
countries. And those plans began to show how 
powerful the experience of the Forum had 
been and how important it may prove in terms 
of improving the practice in different parts of 
the world in the years to come. 


We took some time to assess the experien- 
ces of the Forum, shared a few words, some 
images, some songs, and finally held hands and 
sang “We shall overcome” in several lan- 
guages. We said goodbye, though only for the 
time being, to reunite in the future with a 


common vision, a common perspective, and a 
common hope. 


KEY ISSUES IN 


PARTICIPATORY EVALUATION 


The sharing, the debate and the analysis car- 
ried out during the Forum raised a variety of 
issues that are central to our understanding 
and practice of Participatory Evaluation. 
These issues were identified and discussed; a 
number of stands were taken on these and not 
all were in consensus. The diversity of the 
context and the range of practice represented 
by the participants in the Forum perhaps 
reflected itself in the debate and discussion of 
the issues in Participatory Evaluation. In this 
section these issues are highlighted and the 
multiple positions taken on those issues are 
articulated. This elaboration of the issues in 
Participatory Evaluation can alert the practi- 
tioners about considerations in planning and 
carrying out a_ Participatory Evaluation 
exercise. 


1. The context of participatory 
evaluation 


One of the central considerations raised again 
and again throughout the discussion at the 
Forum was the need to particularize‘a partici- 
patory evaluation exercise ina given context. It 
was felt that the context of the exercise was 
crucial in designing and implementing an eva- 
luation process, and its appropriateness to a 
given context would provide the possibilities 
of effectiveness, or otherwise, of a participa- 
tory evaluation exercise. 


The elements in this sensitivity to the con- 
text ranged from the political and economical 
to the cultural and social. The experience of 
utilizing traditional and folk perspectives in 
the African context reflected the cultural sensi- 
tivity in those efforts; the reference to political 
domination in Philippines was necessary in 
situating any participatory evaluation effort in 
that context; the context of liberation and 
struggle for maintaining freedom and auto- 
nomy in Nicaragua were to be articulated to 
situate any given evaluation intervention; the 
nature of developmental models and philoso- 
phies prevalent and dominant in a society like 
India had to be understood before initiating 
participatory evaluation efforts at the grass- 
roots level. These and many more such exam- 
ples and illustrations highlighted the impor- 
tance of situating a given exercise with specific 
sensitivity to the local context. 


TRENDS IN PARTICIPATORY 
EVALUATION IN AFRICA 


Three distinct participatory evaluation 
approaches seem to be emerging in 


Africa. The first, which can be described 


as the Tanzania school, describes the 
approach which is basically university 


based and is politically activistic. This 


approach is being widely practised in 


— East, Central and Southern Africa and is 


the focus of the Africa Participatory 
Research Network (APRN). The other 
which is a mix of grass-roots and institu- 
tional initiatives is being widely prac- 
tised in Francophone Africa. Anglophone 
West Africa is experimenting with ‘oral 
participatory evaluation’ approaches that 
have been used for centuries by tradi- 
tional African societies. These use folk- 
lore, songs, stories and proverbs to” 
evaluate, and reinforce problem-solving/ 
learning and unlearning activities within 
their communities. They were also used 
in the resistance to alien, external influ- 
ences and to rehabilitate African society 
after the ravages of colonialism. 


However, the African animators, 
coming out of their own societies, used 
these approaches to move their people 
beyond survival and subsistence towards 
self-reliance. The value system became 
constant indicators, whilst new ideas and 
influences became variables in this app- 
roach. Thus the communities were led to 
affirm, negate, resolve, review or change 
communal objectives and direction. 
Regular periodic meetings, such as the 
40 day cycle, of communal meetings— 
the Odwira of the Akan of Ahana and 
the annual harvest festivals—were used 
for collective self-reflected awareness 
and to order the life of the community. 


JAMES ANNORBAH-SARPEI 
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Yet, time and again, the reference to the 


political context of a participatory evaluation 
exercise kept coming up. There were those 
who strongly believed that a given political 
context determined the perspective, the roles 
and the strategy for participatory evaluation 
and without reference to political context any 
participatory evaluation effort may end up 
being controlling and frustrating. [t was 
believed that participatory evaluation cannot, 
and does not, operate in a political vacuum and 
the larger socio-political framework has to be 
considered. This debate led to the second 
issue—the question of the mission and pur- 
pose of participatory evaluation. 


2. The mission and purpose of 
participatory evaluation 


What is participatory evaluation for? Is it 
merely an attempt to assess the impact of a 
given activity through a more creative set of 
tools and techniques? Does it have a broader 
mission in view? It was generally felt that 
participatory evaluation is not merely a techni- 
cal exercise with a set of tools and techniques 
to collect and analyse data and to reach some 
conclusion. It was seen as a political process 
with an overall mission of empowering the 
masses, the poor and the oppressed. This 
overall mission needed to be explicitly under- 
stood and stated such that the design and con- 
duct of an evaluation does not peEencrae into 
some mere technical exercise. 


How does this mission get articulated? How 
does it provide the basis for defining the pur- 
poses of participatory evaluation? The basic 
point about participatory evaluation is its abil- 
ity to help people value their own experience 
and knowledge. The poor masses and the 
exploited sections of society negate, reject and 
undermine their own experience and knowl- 
edge. This is what they have been made to 
believe. It then attempts to re-generate critical 
faculties and reflection capacities of the power- 
less masses so they could analyse their own 
reality and attempt to transform it. It is this 
potential of participatory evaluation which 
helps establish its purpose and provides the 
basis for linking it to its overall mission of 
empowerment of the people. 


In a society full of contradictions, participa- 
tory evaluation helps to strengthen the capaci- 
ties of the powerless masses to recognize and 
confront those contradictions. In our societies, 
where the dominant knowledge has estab- 
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lished its unchallenged priorities over people's 
knowledge, a participatory evaluation exercise 
tends to help people to collate, converge and 


analyse their various experiences in order to 


form an alternative to this knowledge and to 
evolve a perspective that provides the basis for 
transformation. Participatory evaluation thus 
provides the basis for supporting and stren- 
gthening the’ processes of transformation in 
the society. 


The debate in the Forum highlighted the 
pitfalls in undermining this central mission of 
participatory evaluation—to empower the 
people—through a variety of practices that 
degenerate into narrow technical implementa- 
tion of a set of techniques. It was mentioned 
that several stake-holders and constituencies 
accepted participatory evaluation only if it did 
not challenge their status quo, if it did not raise 
fundamental questions about the perspectives 
and philosophy of their work, and if it helped 
to preserve the basis of the existing reality. 
Several examples were quoted where it was 
shown how practitioners coopted some of the 
basic tenets of participatory evaluation in 
order to play the via media role between the 
constituencies and stake-holders, thereby sus- 
taining the status quo. 


In contrast it was strongly argued that par- 
ticipatory evaluation should be used as a tool 
for advancing the vision of people’s empow- 
erment and development, a tool for develop- 
ing their own terms of reference rather than 
working on given terms of reference, a tool to 
promote leadership among the powerless 
people, a tool that helps to link micro actions 
within a macro perspective. 


What then is the nature of this mission? 
How does one understand transformation as 
opposed to status quo? 


3. Reproduction or transformation: 
Habitus or corpus 


Participatory Evaluation is essentially a pro- 
cess of learning an educational process pro- 
moting reflection and critical analysis by the 
powerless and oppressed of their own reality 


and circumstances. This reflection about their 
own heritage and milieu helps them to re- 
create their own knowledge and differentiate 
between natural factors within their own real- 
ity from the socio-political factors. It is a pro- 
cess of separating what is possible and given 
from what is desirable and necessary. It then 
becomes an exercise in that transformation, 


HABITUS. 


CORPUS 


i. Social Ego Question it,not accept it} Intentional ego 
(Relationship of heri-| and transform it. 
tage and milieu with the 7 
subjects which keeps 
getting reproduced or 
keeps perpetuating itself) , 
ii. Affinity Lift self from it. Identity 
(Hold of heritage and 
milieu) 
iii. Systems Critically look at them, | Systems 
(Inherited or provided) | adopt them/ change | (Created or evolved) 
| them/ transform them. 
iv. Existing consciousness | Test it in practice, mod- | Enhanced consciousness 
cs ify it, rectify it, refine it 
-y. Determinism Identify, understand, Liberation 
_ (Leaving things to occur | intervene and_ guide 
on their own) change 
not merely reproduction—it becomes the that they can develop their capacities to trans- 


“corpus”, not the “habitus”’. 


What is “habitus”? The meaning of “habi- 
tus” is to produce and reproduce the existing 
cultural order as represented in ideas, attitudes 
and communication; essentially to continue 
things as they are and to do them better. This is 
the effectiveness model, where empowerment 
within the given frame is attempted through 
participatory evaluation. 


And what is ‘corpus’? “Corpus” implies 
transforming the “habitus”, not accepting it as 
it is but bringing about a qualitative shift, 
including changing meaning of words. It is to 
question and challenge the frame and not 
merely to improve the functioning of that 
frame. 


And what is the meaning of this shift from 
the “habitus” to the ‘corpus’? As illustrated in 
the figure, this shift implies a series of pro- 
cesses which help transform social ego into 
intentional ego, affinity into identity, systems 
inherited or provided into systems created, and 
pre-comsciousness into critical consciousness, 
determinism into liberation. This is the mean- 
ing of the broader mission of participatory 
evaluation—to empower the powerless such 


form their reality. 


This seems to imply that participatory eva- 
luation tends to take sides, it tends to repre- 
sent the standpoint of the poor, the oppressed 
and the powerless. And this implies that sub- 
jectivity is an essential ingredient in participa- 
tory evaluation. 


4. Incorporating subjectivity 


As a shift from the habitus to the corpus is a 
strategy to assist the process of transformation 
as opposed to reproduction, participatory eva- 
luation represents the interests of the poor and 
the powerless. It builds on their existing expe- 
rience and knowledge and helps them to 
develop their capacities to critically analyse 
and understand their situation in order to 
transform it collectively. It therefore becomes 
the: process that is a non-neutral leverage to 
the of transformation of 
society. 


larger process 


It was in this context that participants at the 
Forum discussed the issue of incorporating 
subjectivity in the process of evaluation. Tradi- 
tional evaluation attempts to deny and negate 
the subjectivity of the participants in the pro 
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cess. It attempts to pretend that subjectivity 
distorts representation of reality. Quite to the 
contrary, participatory evaluation seeks to 
incorporate the subjectivity of the participants 
who are expected to transform the reality. A 
series of examples and illustrations were given 
to highlight how incorporation of subjectivity 
in the process of evaluation makes it.a trans- 
forming experience for the participants as well 
as creates the conditions for their empower- 
ment. It is the view-point of the participants, 
their perceptions of reality, their feelings and 
emotions attached with those perceptions and 
their expression of their own experiences 
which represents subjectivity. The discussion 
in the Forum highlighted how important it 
was to deliberately and intentionally incorpo- 
rate and utilize this subjectivity in the process 
of participatory evaluation and that, in fact, its 
denial has been effective in perpetuating the 
status quo. The subjectivity of the participants 
in the process of evaluation is encouraged, 


Viewpoint and standpoint are developed 
on the basis of our theory of knowledge. 


The classic example of the frog in the 
well who thought that the heavens were 
only those that he saw from the bottom 


well and realizethat there was a lot more 
of the heavens, comes to my mind. 


preferred and has continued to be loyal 
to that part of the heavens that he con- 

_tinues to see from the bottom of the well. 
He is a wiser frog, but he has not rejected 
his part of the heavens. 


Another example that comes to my 
mind is that the process of conscientiza- 
tion not unlike that described by Paolo 
Freire where the organizer or the re- 
searcher gathers the new data, the dif- 
ferent perceptions of the individualized 
Masses, systematizes them, similar to 


that process is putting together the pie- 
ces of a jigsaw puzzle. 


Later he presents the complete jigsaw 


EPISTEMOLOGY: WHERE DO CORRECT IDEAS COME FROM 


of the well, until he climbed out of the oe women, and so on. 


Although I must say that the frog still Have always seen the sky outside the 


_demned his fellow- frogs forever 1 to 


-ness and helplessness, from oppression — 


localized and included in the analysis and 
reflection. It is this subjectivity, their own per- 
ceptions and experiences of reality, which pro- 
vides the richness and depth to our under- 
standing and becomes the basis for challenging 
that reality and transforming it. It thus 
becomes the basis for defining, elaborating and 
enhancing the participation of the subjects in 
the process of evaluation. 


5. The issue of participation 


The issue of participation in participatory eva- 
luation is a much abused, confused and 
maligned one. Participation has been seen as a 
way of getting the actors in the process excited 
about the process—getting them involved, 
getting them active. Many a times it is seen as a 
gimmick, as a technique; as a way of finding 
out how people feel and react, as an instrumen- 
tal aspect of the evaluation exercise. 


puzzle back to the scattered masses who 
recognize their reflection, the reflection 
of each of them, galvanizing them intoa 
unified consciousness of a class—the 
workers, the peasants; of a sector— 


So intimate is this process of conscien- 
tization that its betrayal by those who 


frog’s well can lead to destruction and 
new subjugation. Worse still is the bet- 
rayal by the frog who has seen the sky 
from the bottom of the well and has — 
struggled out of it to see the vastness of 
the heavens and has not only rejected 
that part of his heavens but also con- — 


bottom of the well. 


There is, chop fore jrscificade ns i 
statement that participatory evaluatic 
can only be genuine if it is a scientif ic | 
instrument of the organized masses in © 
their bid for liberation from powerless- ; 


and exploitation, from ignorance and 
superstition. NOEL MONDEJAR 


However, participants in the Forum, by and 
large, rejected this position and felt that Partic- 
ipation is the central issue in participatory 
evaluation and a normative one at that. Parti- 
cipatory evaluation builds on the notions of 


centrality of the people in the process of reflec- - 


tion and analysis. An overall mission of 
empowerment of the people is not possible 
unless they remain at the central stage of the 
evaluation process. 


in this context the notion of decisive and 
‘non-decisive participants was based. Decisive 
participants are those who are central to the 
process that is being evaluated. These are the 
activists, the animators, the local people. The 
non-decisive participants are those who have 
stakes in the evaluation process but are not 
central to it. These are the intermediary organ- 
izations, the donors and other stake-holders. It 
is important to recognize that any evaluation 
exercise encompasses a vast constituency 
which has multiple stake in the outcome of 
that exercise. The donors, the resource 
providers, the heads of implementing organi- 
zations, the consultants, the facflitators, the 
intermediary organizations, the grass-roots 
activists and animators and the local people, 
the beneficiaries, the target groups all repres- 
ent a variety of constituencies and stake- 
holders. And this range of stake-holders has 
different expectations and stakes in the out- 
come of the evaluation process. 


This recognition is important to develop the 
process of participation in a participatory eva- 
luation exercise. The issue of participation 
cannot be settled without this understanding 
of multiple stake-holders and multiple stakes. 
Built into this exercise has to be a possibility of 
expressing the viewpoint of all stake-holders, 
their perception of reality, and their hopes and 
aspirations from the evaluation exercise. Yet 


the standpoint of the evaluation has to be of © 


the decisive participants. It is in the interest of 
the decisive participants, the central actors in 
the process under evaluation, the local people 
themselves whose interests must override any 
consideration in the participatory evaluation 
process. Thus, the participants at the Forum 
illustrated, through their experience, how 
maintaining the centrality of the decisive par- 
ticipants in enhancing and strengthening the 
process of participatory evaluation is crucial. 


And it is crucial in respect of a series of 
questions which were raised by the partici- 
pants, which affect the nature, the meaning 


and the character of participation ina Partici- 
patory evaluation exercise. Some of these ques- 
tions are : 

a. When is participatory evaluation to be 
done? How is it decided? Whose needs 
are represented? What is the context of 
the evaluation? 

b. How is the framework and terms of 

. reference of participatory evaluation 
established? How is it decided what to 
evaluate? 

c. And under what conditions do multiple 
stake-holders get involved in the pro- 
cess? How does one distinguish between 
manipulation and participation? How 
does involving multiple constituencies 
not sidetrack the centrality of the deci- 
Sive participants? 


The debate and discussion in the Forum 
brought out a variety of ways by which critical 
reflection at the beginning of the exercise was 
seen as an important ingredient in determin- 
ing and ensuring effective participation in a 
participatory evaluation process. Several 
approaches were presented, one of which is 
highlighted in the box. It was generally felt 
that unless serious, careful and critical atten- 
tion is paid to these questions of participation 
at the early stages of planning a participatory 
evaluation process, it may be difficult to ensure 
the centrality of the decisive participants and 
the overall mission of the participatory evalua- 
tion process. Fears were expressed about high- 
jacking a participatory evaluation exercise 


Actors : Who are the actors in 
the PE process? What 
roles do they play? 
What responsibilities 
do they undertake? 


Situation : What is the situation 
and context on which 
PE is situated? 


Individual Group : What are the relation- 
ships between the in- 
dividual and _ the 
group? 

Conscientization: Who are we em- 
powering; individu- 
als, groups? 


Process : Who initiates the PE 
process? What for? 
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away from the decisive participants, away 
from the standpoint of the powerless and the 
oppressed, away from the overriding mission 
of empowering the powerless, if careful atten- 
tion about identifying and structuring the par- 
ticipatory process is not paid in the early stages 
of an exercise in evaluation. 


6. Factors in authentic participation 


Time and again participants at the Forum 
deliberated about various factors that seemed 
to contribute to or obstruct authentic partici- 
pation of various categories of participants in 
the process of participatory evaluation some of 
these are highlighted here: 

a. The existing level of literacy among the 
participants can influence their partici- 
pation in the process. This can be par- 
ticularly so if evaluation uses data- 
collection methods like surveys and 
questionnaires. However, if other 
methods are used which are symbolic 
and expressive like drama, role-play 
and songs then participants with lower 
levels of literacy can also be involved in 
the process effectively. 


b. Availability of proper and sufficient 
time is a crucial factor in generating and 
sustaining authentic participation. In 
setting up the process of participatory 
evaluation, a considerable amount of 
time is spent in sorting out problems, 
raising issues, resolving conflicts, get- 
ting to know the people, etc. However, 
many a times sufficient time is not 
allowed in the early stages of preparing 
the ground; as. a result, participation 
gets distorted and in many cases insuffi- 
cient time spent in structuring the pro- 
cess can leave the decisive participants 
out of the process. Concerns were raised 
about the fact that important stake- 
holders like donors and heads of organi- 
zations do not allocate sufficient re- 
sources and energies and thereby 
undermine this crucial factor of the 
time necessary in the process. 


c. It was felt that sufficient groundwork 
needs to be done to facilitate the reflec- 
tion and discussions amongst the deci- 


Sive participants. This preparatory work 
becomes all the more important for 


those participants who have very little 
experience or opportunity to ask critical 


questions, to enter into dialogue and to 
debate issues. Preparation of this 
groundwork will entail encouragement 
and development of an environment 
which is non-threatening and psycho- 
logically safe and supportive. 


d. Sometimes excessive thrust on data- 
collection, almost an obsession with it, 
can hinder the process of participation. 
Involvement of people in a significant 
and central way should move beyond 
the constraint set up by quantitative or 
qualitative techniques and methods of 
data-collection. 


e. Illustrations were given where use of 
words and phrases and symbols tended 
to confuse and inhibit participation 
rather than clarify and enhance under- 
standing. It was felt that words and 
symbols familiar to the people, and not 
alien to them, should be utilized in this 

. process. 


f. Inorder to generate sustained, effective 
participation throughout the process of 
participatory evaluation, it was felt that 
effective facilitation is necessary. This 
facilitation should not be either over- 
powering or dlienating. 


g. Participants in the Forum repeatedly 
talked about the need to develop a 
perspective on participation and how 
to use it as a signpost at every step of 
the process. It was felt that abuse of 
participation without understanding its 
full implications and perspectives can 
distort the process and inhibit the 
movement of participatory evaluation. 


7. The gender issue 


Related to this question of participation was 
the issue of gender in participatory evaluation. 
Participants in the Forum repeatedly high- 
lighted the importance of taking gender into 
consideration in elaborating the _ political 
framework and specifi¢ terms of references; 
and designing an evaluation exercise. It was 
felt that an authentic analysis in participatory 
evaluation cannot occur without taking cog- 
nizance of, and giving due value and impor- 
tance to, the issue of gender. It was generally 
felt that male perspectives still dominate the 


onceptualization and practice of participatory 
valuation right from setting up the initial 
rame of references to data-collection to analy- 
is and so on. It was felt gender issues are taken 
or granted, and specific debate highlighting 
pecific concerns of women and particularities 
f their situation is not made the central focus 
n-such exercises. It was felt that women’s 
erspectives and issues should be explicitly 
acorporated and integrated in the process of 
valuation and that they should not be sub- 
umed under the general category of the pow- 
rless and the exploited. It was reaffirmed that 
long with other contradictions in the society, 
he framework of the context of participatory 
valuation must recognize the patriarchal con- 
itions existing in society and the oppression 
f women in the structure of society. It was felt 
hat the issue of transformation should specifi- 
ally address itself to the transformation of 
atriarchal structures which oppress women 
yithin the society, in our own organizations 
nd our own lives. 

A related issue was the question of sensitt- 
ity, in the process of facilitation, to the needs 
nd concerns of women during the process of 
valuation. It was felt that women facilitators 
hould be encouraged and provided central 
oles in such exercises and that male facilita- 
ors should be sensitive to the participation of 
yomen and their needs and concerns in the 
rocess of evaluation, as well as those of their 
omen colleagues. 


Through illustrations from a variety of 
tuations, the participants at the Forum high- 
hted’the importance of explicitly recogniz- 
g and articulating the issues of gender in a 
rticipatory evaluation exercise. It was felt 
at continuous and deliberate attention needs 
be paid to gender issues if we are to enrich 
t only the analysis ‘of reality but also the 
rategies to transform the same. 


Structuring the process 


e debate in the Forum highlighted the 
arkable diversity and context-specifity in 
variety of structures utilized in a given 
rticipatory evaluation effort. It was felt that 
prescriptions can be made on how to go 
ut designing and structuring a participa- 
evaluation effort. These will have to be 
eloped within a given context in considera- 
n of a variety of factors enumerated here 
elsewhere. However, it was felt that cer- 
steps in the process of participatory eva- 
tion are usually common, though their 


actual form and elaboration may vary from one 
context to another. As a consequence of the 
sharing and reflection generated in the Forum, 
the following steps seem to be the key stages in 
a participatory evaluation cycle : 


a. Thecurrent experience of participants 
This becomes a starting point in the 
process of articulating the need for par- 
ticipatory evaluation, setting its objec- 
tives, with special reference to the con- 
text in which the exercise is to be 
undertaken. 


b. Establishing the frame of reference 
and planning the details of the process 
It is here that the concerns of the deci- 
sive participants need to be kept central 
while ensuring the participation of all 
the stakeholders. 


c. Data-collection and consolidation 
Here the choice of methods becomes an 
important one. 


d. Analysis 
This is the stage of systematizationof 
the information obtained in ways and 
categories that lend themselves to an 
understanding of the participants. 


e. Feedback 
This has to be provided to the various 
stake-holders, in particular to the deci- 
sive participants. 


f. Decision making for action 
On the basis of the feedback in taking 
actions for transformation. 


g. Realization 
Leading to deeper experience of the 
participants. 


Thus these steps tend to constitute a cycle 
and participatory evaluation becomes a cyclical 
process with each layer of experience and 
reflection deepening our understanding and 
sharpening our actions. 


9. Methods 


It is in this context of designing and structur- 
ing the process of participatory evaluation that 
the choice and utilization of methods of data- 
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collection becomes important. Participants in 
the Forum spent considerable time identifying 
factors which make a method appropriate and 
effective. It was generally felt that methods 
should be so chosen that they are liberating 
experiences and not subjugating experiences, 
that they involve the people and enhance their 
conscientization. The debate also highughted 
the need to look at the methods as vools tor 
identifying and elaborating information and 
not as the representation of the totality of the 
process of participatory evaluation. The dis- 
cussion identified several key questions that 
could be asked in respect to choice of methods. 


a. Who owns the method? 
Who will own the methods? Will it be 
limited to the facilitators? Would dif- 
ferent categories of participants be in- 
volved in the process of selecting and 
using various methods? 


b. intent 
What is the intent and purpose of dif- 
ferent methods? What type of data is 
needed? Will different methods be 
necessary for different types of data? 
c. Culture appropriateness 
Are the methods relevant to and fitting 
into the culture of the participants? Are 
they alien to the culture of the decisive 
participants? Are certain methods more 
appropriate and easily understood and 
useable than others? 


d. Facilitative 
Do the methods facilitate learning? Do 
they help people take control and feel 
empowered? Do they help articulate 
subjectivity of the people? Do they help 
them to value and express their own 
knowledge and viewpoint? 

e. The use 
How are the methods used? Are they 
used in a way which is sensitive to the 
various stake-holders? Are they used 
for data-collection only or to generate 
interest and mobilize the participants 
in the evaluation process? 

f. Qualitative versus quantitative 
Which type of data is necessary? In 
situations where qualitative informa- 
tion and subjective assessments are 
more important, which type of methods 
can bring out that data? Should we limit 
ourselves to one type only or should we 


be flexible, depending on the type of 
information needed? 
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g. Gender sensitive 
Are methods sensitive to the special 
concerns and needs of women? Are 
they able to focus and highlight gender 
specific issues? 


h. Training 
Are the stake-holders involved in the 
evaluation process helped and trained 
to use the methods? How much energy 
and time is spent in providing this 
training? 


The discussion on methods also focused on 
choosing methods on the basis of their appro- 
priateness and relevance in a given participa- 
tory evaluation exercise. It was felt that within 
the same exercise, depending on the consti- 
tuency of the stake-holders from whom the 
information is being collected, different 
methods can be utilized. Thus, methods of 
data-collection with the community of beneft- 
ciaries put focus on use of drama and role-play, 
songs, stories; in-depth interviews could be 
used with members of the organization; struc- 
tured questionnaire could also be used to elicit 
information from donors and others with 
whom the organization comes in contact. 
Depending on the nature of the organization, 
whether it is an intermediary organization, a 
grass-roots organization, an organization 
which has a long history, or a recent organiza- 
tion, appropriate methods of data-collection 
have to be chosen. Besides, the existing culture 
of the organization where such an exercise 1s 
being undertaken may also limit the choice of 
an appropriate set of methods. Thus, selection 
of methods needs to be a careful effort, and the 
quality of information generated will necessar- 
ily depend on how appropriate these methods 
were. 

Participants in the Forum also discussed 
various sets of methods, highlighting their 
potentials and drawbacks, as per their own 
experiences. These are shown in the box. 


10. Facilitation 


It was generally felt that every participatory 
evaluation process requires facilitation of one 
kind or another. In order for the process to be 
designed, structured, and sustained over a 
period of time, some degree of facilitation is 
needed. The question was on the issue of who 
facilitates and how. The debate in the Forum _ 
claritied the distinction between the role of 
facilitation and the person of the facilitator. It 
was felt that the importance and significance 
of facilitation per se should be understood and 


POTENTIALS AND DRAWBACKS OF METHODS 


Potential Drawback 
1. Survey produces quantitative data written word can be 
Questionnaire from large number of sour- alienating, dependence on 


ces, increases skills of com- _ literacy/ numeracy levels 
munity, group questionnaire | 

ul. Interviews existing information drawn lot of time and resources, 

out,can generateenthusiasm suspicion, can suffer en- 
thusiasm 


ili. Discussions informal, can be more facili- leader/a few domination 
Meetings tative | 


iv. Pictures good qualitative data, can concretization and system- 
Stories facilitate participation, im- _atization may be difficult, 
Songs mediacy of impact,enhances takes time and training, 
Puppets participation can be expensive 
Photographs 
Slides 


v. Drama/role-play conceptual, attitudes gra- possibility of bias 
| phically portrayed,identifi- 
cation strong 


vi. Case study historical process can be 


shown 


examined separately from the question of who 
is the facilitator. 


An effective facilitation of a participatory 
evaluation process seems to serve several 
functions. Facilitation was needed to creaté an 
environment of sharing and reflection, of 
encouraging participation and expréssion of 
people's experience, knowledge and view- 
points. Facilitation was necessary to build the 
confidence and the capacity of the people to be 
an active owner and partner in the process of 
evaluation. Facilitation was also necessary to 
stimulate critical faculties of analysis and reflec- 
tion among various constituencies, in particu- 
lar the deciseve participants. Facilitation was 
necessary to hetp identify resources and ways of 
collecting information, and analysing and sys- 
tematizing that information, and planning 
actions to help bring together the viewpoints 
of various stake-holders in a given participa- 
tory evaluation exercise, and to facilitaté a dia- 
logue between the constituencies on the issues 
raised by the evaluation. 


It was felt that facilitation should not subju- 
gate, overpower, threaten or alienate partici- 
pants in the process. It should not distort the 
opinions or perspectives of different consti- 
tuencies. It should not overlook the socio- 
political context and the specificities of rela- 
tionships in’a given exercise. It should not 
negate or gloss over the gender issues. It 
should not hide the viewpoints of the facilita- 
tor, but attempt to incorporate them in the 
process. 


Effective skills of facilitation meant a var- 
iety of things. It meant sensitivity to the view- 
points of different stake-holders. It meant 
understanding of the context in which the par- 
ticular evaluation is being undertaken. It 
meant identifying decisive participants and 
enlisting their central role in the evaluation 
process. Facilitation also meant facilitating the 
process of evaluation, of setting one stage after 
another, for collecting, analysing and utilizing 
information. Building relationships with dif. 
ferent constituencies was an effective skill in 
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facilitation. Maintaining a balance between 
multiple viewpoints represented by multiple 
constituencies was a key facilitation compe- 
tence. Sensitivity to the individual and group 
styles, expectations and orientations was also 
seen as an effective facilitation competence. 


The question of facilitation by whom was 


seen as a difficult one. First of all, a set of 
persons having the competence of playing the 
various facilitator roles was necessary. These 
persons could be from within the context, they 
could be outsiders; they could be a combination 
of insiders and outsiders; but the important 
thing was that they had a set of competencies 
necessary to effectively play the role of facilita- 
tion and that they were accepted by the various 
constituents and stake-holders in the evalua- 
tion exercise. The second aspect of the ques- 
tion is the issue of expertise. It was felt that 
expertise in participatory evaluation is essen- 
tially the expertise in facilitation skills. It is 
not necessarily the expertise of the content. 
‘Hence it is possible to utilize facilitation and 
content expertise both.from within the setting 
as well as from outside, should it be necessary 
in a given evaluation efforts. But leaving the 
entire facilitation role to such experts is to 
undermine the role of facilitation. 


Participants generally felt that one of the 
overall objectives of a participatory evaluation 
exercise should be to help transfer and streng- 
then facilitation skills among various consti- 
tuencies, in particular the decisive participants 
such that it could enhance their self-reliance 
and continuing self-reflection and evaluation 
as an ongoing activity. Similarly it was felt that 
part of the focus in a participatory evaluation 
exercise should include evaluation of the facili- 
tation and the facilitators. This will ensure 
accountability as well as an opportunity for 
reflection and learning about the role of 
facilitation. 


The debate highlighted the importance of 


facilitation in participatory evaluation in many 


ways. It was felt that nothing could undermine. 


its importance, despite the hurdles and the 
problems that are faced. Therefore, ensuring 
effective facilitation throughout the participa- 
tory evaluation process was seen as a critical 
issue in-undertaking such an exercise. 


11. The issue of unequal power of 
stake-holders 


One of the recurring themes in the discussion 
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on participation and the structuring of the | 
process of participatory evaluation was ways | 
and means needed to cope with unequal 
power-relations between various stake-holders 
and constitvencies in a given exercise. While 
identifying various constituencies in a given 
participatory evaluation effort, different sets 
of participants have different power relations 
among themselves. For example the local peo- 
ple may feel dependent on the animators, the 
animators may feel powerless in front of the 
heads of their organizations; the organization 
as a whole may feel powerless in the face of the 
donor agencies. If the sensitivity to this un- 
equal power relations is not maintained, the 
process of effective and authentic participation 
may be threatened and a participatory evalua- 
tion exercise may degenerate into an exercise 
controtled by the most powerful. 


In this context, special debate took place in 
relation to the role of the donors. It was gener- 
ally accepted that donors of a programme or an 
organization are legitimate stake-holders to 
the outcome of a participatory evaluation 
exercise. Their viewpoints need to be incorpo- 
rated in the process. But the question of stra- 
tegy becomes crucial in terms of how and 
where we incorporate their viewpoints. Care 
has to be taken that the need for future funding 
does not distort the process of participatory 
evaluation and undermine critical reflection 
by decisive participants. 


In this context some references were also 
made to the need for focusing participatory 
evaluation efforts at the donors and their 
organizations. It was felt that this kind of pro- 
cess will ensure mutuality in reflection and 
create the possibility for a more mutually sup- 
portive and just relationship between donors 
and programme people. By and large, the issue 
of careful structuring of participation and 
incorporating viewpoints of constituencies 
with unequal power relationship was under- 
scored. | 


12. The validation of the participatory 
evaluation process 


An important and interesting issue related to 
the question of how does one validate a parti- 
cipatory evaluation exercise. What are the 
bases by which validation of participatory eva- 
luation occurs? What are the indications for 
assessing validation? Are there some impor- 
tant dimensions in this validation which need 
to be understood at the very start of the 
process? 


Participants in the Forum deliberated on 
this issue at some length and some interesting 
aspects on validation of a participatory evalua- 
tion exercise were presented. 


It was mentioned that if. PE is a collective 
process of reflection and action on both the 
past and the future, then it helps the group to 
critically reflect upon its primary objectives, its 
goals, its process and how empowerment of 
the marginalized has taken place. Thus, it 
brings to the group’s attention whether their 
plan of action has been effective and meaning- 
ful, or not. In such a process PE is valid if it 
brings to the consciousness of the group their 
actual realities and thus helps bring about and 
catalyze change at all levels of functioning. 


The question that logically follows is: How 
does one arrive at the indicators of this change? 
If we predetermine the indicators then the 
assumption is that we are predetermining the 
outcome. Conversely, if we perceive change as 
a dynamic process then one cannot predict the 
outcome. In the former, the process becomes 
manipulative leading towards “expected” goals 
or results. 


However, it was discussed that PE processes 
have certain inherent value-systems which are 
reflected in the overall goals and objectives; 
which are premised upon the fact that the 
oppressed have to act on the basis of their 
collective strength to liberate themselves from 
the exploitative structures. Thus the PE pro- 
cess has within itself certain value-based 
indicators. 


Conventional evaluation efforts or evalua- 
tion carried out on behalf of certain stake- 
holders (like donors) anticipates and progres- 
ses upon a harmony approach, leading to the 
maintenance of the status quo, through mere 
incremental improvements. PE, on the other 
hand, highlights the inherent contradictions 
among and within the various constituencies. 
If this “highlighting” leads to a questioning of 
assumptions, then this in itself is a validation 
of PE. 


The PE process helps highlight not only 
individual strengths, capabilities and limits, 
but also attempts to harmonize intergroup 
relationships, the focus of PE is not on 'per- 
sonal salvation” at the expense of social libera- 
tion. As also, PE diagnoses organizational 
strengths and weaknesses through a collective 
process by the members themselves, further 
action on these diagnoses is needed. This 
further action establishes the chain of PR, PT 


and PE and thus justifies the wholistic and a 
continuum approach of PE. 


Therefore, from the above, what validates 
PE is the process itself leading to change: indi- 
vidual, group, organizational, social-economic 
and political. 


In certain evaluation processes, where the 
facilitator imposes a set of analysis on the 
group, it may lead to a temporary change in the 
group in order to meet their immediate ends or 
needs, but they have not been internalized by 
the process—this is temporary validation. We 
need to distinguish between different types of 
change : 


Superficial versus Permanent 
Cognitive versus Behavioural 
Visible versus Invisible 

Tangible versus Intangible 
Individual versus Collective 
Conceptual versus Operational 
Ideological versus Organizational 


It is this distinction which helps us in under- 
standing the validation. We have to emphasize 
and reiterate the diagnostic value of PE. Yet 
another validation of PE lies in the belief and 
practice of PE that during the elaboration and 
the implementation of the process, skills and 
capacities are built within the group by the 
facilitators, such that it becomes an ongoing 
process, in contrast to conventional practices, 


“where dependency is deliberately created on 


the outside facilitator thus negating the very 
premise of PE. The validation of PE lies in the 
process of PE which demystifies evaluation and 
shifts control of the methods to the group. 


The process of PE leading to empowerment 
and liberation: of the powerless is its major 
validation. 
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CASE STUDIES 


WOMEN’S DEVELOPMENT PROGRAMME 


Rajasthan, India 


Introduction 


This is a report of an attempt to review the 
Women’s Development Programme (WDP) 
of the Government of Rajasthan. It covers a 
period of 18 months, beginning with the first 
training programme of 24 ‘sathins’ (in Jaipur 
district) in August 1984 to February 1986 


when six districts had been covered with the - 


appointment of nearly 250 women functionar- 
ies. It is our contention that the main feature 
which has contributed to the strength as well 
as the cautious ‘speed’ of the programme is its 
evaluation and monitoring system. Evaluation 


in this context is interpreted as ‘shared reflec-. 


tion’ which is woven in each of its processes, be 
it a training programme or a village meeting 

-or a workshop for subsequent planning. The 
process of monitoring, in this light, becomes 
one of ‘revising’ the strategies in view of the 
freshly acquired knowledge through group- 
reflection. These two features, in turn, give a 
concrete content to the notion of 
‘participation . 


Background: Aims and objectives 


WDP, as an announced programme of the 
Government of Rajasthan, came into being in 
the context of many existing development 
schemes. It registered a clear recognition that a 
fresh approach to women’s development was 
needed. Despite the fact that serious attempts 
were being made to divert ‘benefits to women, 
it was apparent that their condition had con- 
tinued to be that of subjugation and depriva- 
tion. WDP’s point of departure can be seen in 
making a shift of attention from ‘delivery 
mechanisms’ which so far had looked at women 
as objects of compassion and welfare, to a 
serious attempt to strengthen the so called 
‘recipients’. It meant a revised perspective in 
which the running assumption was that of 
viewing women as equal partners with men in 
the process of development and assisting in 
removing all such blocks as prevent the mant- 
festation of this sense of equality. It also took 
note of the fact that men had been entrusted 
with the responsibility for women's develop- 
ment in the family, government and society far 
too long. It was felt that a shift was necessary 
to entrust these responsibilities to women at 


all levels. Putting these two approaches to- 
gether, the principal aim of WDP was seen as 
an attempt to empower women through a pro- 
cess of education and training, communication 
of information, and collective action in order 
to bring about a change in women’s social and 
economic status. 


Organizational structure of WDP 


The following organizational structure was 
envisaged for WDP in early 1984. This con- 
tinues to be the formal pattern around which 
WDP weaves its activities. 


a. Village level 


Each selected gram panchayat has a 
trained village level worker called a 
‘sathin’ who belongs to one of the vil- 
lages of the gram panchayat. She is 
responsible for the formation of 
women’s forums at the village level. 
She works in close linkage with nine 
other sathins of neighbouring gram 
panchayats. The cluster of 10 gram 
panchayats with 10 sathins is co-ordi- 
nated by one ‘pracheta’ 


b. Block level 


The pracheta, a block level government 
functionary, provides support and gui- 
dance to the sathin. She also provides 
the communication link with the dis- 
trict level. 


c District level 

i. At the district level, the District 
Women’s Development Agency 
(DWDA) has been placed under the 
chairmanship of the District Col- 
lector. Each DWDA has a project 
director who is assisted by a project 
officer. | 


ii. Technical resource support is pro- 
vided at the district level by the 
Information Development and Re- 
source Agency (IDARA). This is a 
voluntary agency working in the 
field of adult education and rural 
development. 
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d. State level 
i. The District IDARAs are coordi- 
nated by the state IDARA. 


ii. The Director of WDP is the overall 
incharge of the programme. 


iii. The monitoring and evaluation of 
the programme is facilitated by the 
Institute of Development Studies 
(IDSO), Jaipur. 


Designing a monitoring and 
evaluation system 


A review of earlier experience in development 
interventions showed that generally monitor- 
ing and evaluation had been the most neg- 
lected areas of the development programmes. 
In a few cases, in which evaluation had been 
undertaken, it had been an ‘external’ activity, 
carried out by professional researchers either 
at the end of a development programme or 
after periodic intervals which were fairly long. 
These studies had been more in the nature of 
an ‘audit’ —the emphasis being on finding out 
whether the programme ‘worked’. Since the 
evaluation studies were regarded as fault- 
finding exercises, they were perceived by the 
functionaries as being a ‘threat’ to them and, as 
a result, the findings were ignored. Also, the 
emphasis of these evaluation studies tended to 
be on the quantitative dimensions of the 
programmes. 


Salient features of the monitoring and 
evaluation system for WDP 


The monitoring and evaluation systems were 
seen as complementary processes their aim 
being to ‘improve’ the programme. Evaluation 
was envisaged as an essentially internal pro- 
cess to be undertaken by the functionaries 
themselves as well as the community of 
learners. It was also considered to be an 
organic process of learning. It aimed at the 
functionaries experiencing the importance of 
evaluation in their everyday work so as to 
internalize the process and become conscious 
of its worth. When viewed thus, evaluation in 
their everyday work became 4 process whereby 
the functionaries examined their own expe- 
riences, analysed and understood what had 
happened in the programme and placed a 
‘value’ on the outcome of the work. This meant 
that the functionaries identify their own areas 
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of strength and weakness. It was thought that 
such an evaluation could also help those 
involved in the programme to measure the 
results achieved against the goals they had set 
for themselves. When done collectively, this 
process would become a process of shared 
reflection. Monitoring was defined as a process 
of watching periodically the progress of WDP 
with a view to taking corrective measures in 
order to optimise the effectiveness of the pro- 
gramme. Evaluation was defined as a process 
of collecting evidence and using it to judge the 
degree to which the objectives of the pro- 
gramme were achieved. This assessment, in 
turn, was to form the basis for subsequent 
decision-making in the programme. 


The important characteristics of such a sys- 
tem were summarized as : 
ike Evaluation as ‘a process of reflection 
meant an honest stock taking by all 
those involved in order to ascertain 
what had been achieved, what had not 
been achieved and for what reasons. | 


2. The emphasis in this evaluation was on 
quality rather chan on quantity, on pro- 
cess’ rather than on meeting ‘targets. 


3. Evaluation was a two-way process. 
Information did not merely flow from 
bottom to top, but both ways. In other 
words, ‘feedback’ for corrective action 
was an important component. 


4. Evaluation was participatory. This 
entailed a minimizing of hierarchical 
relationships. It meant listening to 
others, respecting, sharing and learn- 
ing from one another. 


ivaluation and monitoring required a focus 
o« : (i) nature of the programme, (11) process 
0. judgement; (ii1) evidence, (iv) degree of 
achievement/shortfall and (v) decision- 
making. 


The first point emphasized the need for a 
clarification of what to evaluate. The clearer 
the perception and understanding of what was 
to be evaluated, the bettet were the chances of 
the evaluation exercise being meaningful. In 
the context of WDP, women’s development 
was considered the focal point of evaluation 
and it was envisaged that women’s develop- 
ment would be possible if women could (a) 
move from total acceptance. of the present 
situation to a point of questioning it and seeing 
it as a problem which needed to be solved, (b) 


move from the ‘I’ to the ‘we’, i.e. begin to 
realize that it was not a personal but acommon 
problem, (c) begin to examine the alternatives 
which were available, (d) arrive at a decision 
by consensus, (e) evolve leadership for within 
and take collective action. 


It was envisaged that evaluation of WDP’s 
impact on village women would need to recog- 
nize all the activities (or creation of conditions 
for such activities) which generated co- 
operation/association, enthusiasm, learning 
desire, strengthened tools of perception and 
analysis. This had to be seen not only at the 
village levels but at the block, district and state 
level of WDP. 


The second point involved a process of 
judgement. The question to be asked was : 
‘what perceptions were to be taken as impor- 
tant’ ? This was in view of the fact that there 
existed a stereotype of an evaluation exercise 
which had a form of being ‘scholarly’ or 
‘serious’. It led to a trivializationof the more 
significant and meaningful features that 
appeared ‘ordinary’ or ‘trite’. For example, the 
fact that if women had started asking ques- 
tions, overcoming their shyness and lack of 
confidence, it may not have been regarded as 
an important indicator of change or develop- 
ment. It was recognized that in WDP constant 
efforts had to be made to pay special attention 
to this aspect of evaluation so that functionar- 
ies at all levels would become more sensitive to 
the simple but significant perceptions of 
change. 


The third element was that of evidence. 
This meant collection of relevant information 
and data in order to arrive at a dependable 
judgement. This, in turn, entailed an identifi- 
cation of data sources, application of different 
techniques (e.g., written, oral methods, obser- 
vations, interviews, etc.) for data collection, 
both quantitative and qualitative, and an inter- 
pretation of data. 

The fourth element was closely connected 
_with evidence and its interpretation which 
would give an indication of the degree of 
achievement or shortfall in relation to objec- 
tives. This process of judgement would involve 
a continuous reassessment. For example, at 
one stage it may be a positive indicator if 
women’s groups showed a strong desire to sing 
and dance and come together. At another 
point, this very feature would become a neg:'- 
tive indicator if the women’s groups made 1° « 
repetitive monotonous feature at the cost of 


possible learning opportunities. 


Lastly, evaluation had to be used for differ- 
ent types of decision making, such as increas- 
ing or decreasing the involvement of resource 
persons providing new information inputs, 
seeking effective co-ordination between 
development agencies, etc. 


Since evaluation was envisaged as an essen- 
tially internal process that was to be under- 
taken by the functionaries at all levels as well 
as by the village participants, it was essential 
that the channels of communication should be 
such as to facilitate participation and learning 
from one another. Also, it was necessary to 
ensure prompt feedback action. Various forums 
of evaluation were designed for this purpose. 


Forums of evaluation 


These were planned at the village, district and 

state levels. At the village level it was felt that a 

‘jajam’ or meeting would be held every month 

and would involve one pracheta and 10 sathins. 

The objectives of holding a jajam every month 

would be :-— 

1. to give opportunity to the sathins to 
talk about their work and share their 
experience; 

2. to enable sathins to air their problems 
and understand cornmonalities; 

Ds to enable sathins to remain in touch 
with one another and build their 
solidarity; 

4. to provide an opportunity to the pra- 
chetas to collect basic information from 
the sathins about the programme; and 

5: to enable the pracheta to provide feed- 
back to the sathins in the problems 
identified by them. 


Likewise, it was envisaged that a ‘shivir or 
camp would be held every third month in each 
district and would involve 20 - 30 sathins, 
prachetas and WDP officials. Besides the jajam 
and shivir at the village level, other forums 
were created at the district and state levels. A 
monthly pracheta meeting at DWDA. was seen 
as a necessary formal get together of prachetas, 
IDARA and DWDA personnel to provide an 
opportunity for critical reflection on the past 
month's experiences. Likewise, |IDARAs. were 
to organize periodic workshops to provide an 
academic input on any of the development 


issues identified in the meetings. At the state 


level, the state IDARA and IDS(J) were to con- 
solidate the reports from the districts and filter 
relevant issues for holding seminars/workshops 


with national experts. These reflections were 
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to be relayed down by state IDARA to the 
village level in an appropriate idiom for facili- 
tating necessary action. 


Participatory Evaluation : Some opera- 
tional details 


Reflection and analysis have characterized 
every stage of all training programmes for 
WDP. For example, if a role-play is done on 
any issue, participants sit together and analyse: 
what happened, why it happened, what were 
its implications. This kind of questioning takes 
place all the time—whether it is for doing 
exercises together, going through a simulation 
experience or seeking information on health 
or law or on government schemes. Besides 
this, there is an overall appraisal of what each 
participant perceived the training to be. This is 
done not only through reporting daily on the 
event, and learning of the previous sessions, 
but also a summative evaluation at the end of 
the training programme. This evaluation in 
turn, has always been.the background material 
for a post-training workshop which ultimately 
becomes a planning exercise for the next train- 
ing programme. This is how sathins and pra- 
chetas have evaluated the training programme. 


Trainer’s Diary : Jobner 


‘We sat in a circle on the floor on the first 
day of the training at Jobner. Everyone 
thought it might be a good idea to begin 
with a prayer. Why did we want to pray ? 
Everyone looked expectantly at the ‘official 
trainers’. The official trainers looked at eve- 
rybody else. It was suggested that each one 
would state why she wanted to pray. It took 
an hour and a half to decide that we would 
pray for expressing solidarity, for peace, for 
ourselves. We also decided we would like to 
sing together. We did not want to sing a 
religious song. So, we sang a song celebrat- 
ing mankind, hoping that one day we would 
succeed in all that we set out to do.” 


“Later, Madhu was a bit fed up with this 
process. Every single decision had to be dis- 
cussed, each individual opinion elicited and 
then a decision taken. She wanted to know 
why we went through this process at all. 
Couldn’t the few responsible trainers decide 
quickly in the interest of all concerned?” 


One of the main forums of evaluation and 
monitoring continues to be the jajam. The 
following excerpt from a District IDARA des- 
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cribes the pattern of how a jajam is being 
planned. 


“Sathins take total responsibility for plan- 
ning a jajam from the physical arrange- 
ments to identifying the issues to be dis- 
cussed. Once they identify the issues, they 
contact the pracheta and (sometimes) write 
to IDARA to ask for specific information. 
At the jajam, the sathins share their expe- 
riences of the last jajam. Effort is made to 
see how far the last jajam’s decisions have 
been followed through and if not, what ts 
hampering the progress, and what should 
be done. 


‘Working in the villages, sathins often 
have to face problems. Sathin’s role is a very 
crucial one and often she faces vehement 
Opposition, sometimes even harassment. 
At times, even her own family members try 
to thwart the emerging leadership traits 
within her. Many of her sathins have been. 
beaten up for relating to the lower caste 
women. Some have received serious injuries 
while others have been threatened. These 
issues are taken up and the groups attempt, 
at their own level, to tackle the situation, 
varying from a small group of women 
approaching those who harass the sathins 
and trying to reason with them, or resorting 
to legal action. This is not just for the 
sathins. Even when a woman in their vil- 
lage faces this kind of a problem, the sathin 
of the area intervenes and sometimes the 
case is taken up at the jajam level.” 


(FROM IDARA NOTES, JAIPUR DISTRICT, 
JUNE 1985) 


The current reports on jajams show that 
three-four major issues come up repeatedly. 
These are : 


i. Health issues : These come up in some 
form or another in every jajam. 


ii. | Water : There is either a shortage of 
water and/or wells/handpumps, or’the 
handpumps are out of order. 


iii. Lack of information on 
a. different development schemes; 
b. welfare programmes; 
c. legal aid assistance and legal matters. 


iv. Problems of organizing women regard- 


ing injustice in wage disbursement as 


well as availability of grain. 


Over a period of time, other modes of eva- 
luation have evolved. These are : 
i. informal letters, 
ii. pracheta monthly meetings, 
iii, training and re-training sessions, 
iv. workshops organized by IDARA and 
Institute of Development Studies, Jaipur. 


Approximately, one workshop every two 
months has been an overall feature for the past 
one and a half years. 


Issues in participatory evaluation 


The previous section has described how the 
evaluation system for WDP was designed and 
the manner in which the evaiuation system has 
become operational. 


In this section, an attempt is made to syn- 
° ° ‘ ° 
thesize issues that get highlighted because of 
the process of participatory evaluation. 


1. Demystifying evaluation 


Since participation is central to the function- 
ing of WDP in all respects, the process of par- 
ticipatory evaluation follows as a natural corol- 
lary. Due to the fact that functionaries have 
internalized and operationalized participatory 
modes of functioning, their active involve- 
ment in the evaluation process becomes a 
somewhat easier task. What this means ts a 
‘demystification’ of the concept of evaluation 
so that the functionaries recognize its impor- 
tance in their everyday work. 


2. Control 


A related issue is that of ‘control’. Tradition- 
ally, the outside evaluator has exercised contro! 
over the entire evaluation process. In the case 
of participatory evaluation, however, this con- 
trol is exercised not by the professional evalua- 
tor but by the people whose work is to be 
evaluated. In WDP, this realization in itself has 
led to greater participation and involvement of 
the functionaries in the evaluation process. As 
a result, more novel modes of evaluation have 
evolved over a period of time. 


3. Enhanced awareness 


Since reflection and analysis are integral pro- 


cesses of WDP evaluation, the awareness level 
of village women and of WDP functionaries 
has been greatly enhanced. While the aware- 
ness levels vary across individuals and groups. 
the fact remains that collective analysis and 
reflection result in higher levels of awareness. 


4. Empowerment 


As awareness levels go up and new learning 
takes place or new knowledge is sought, a feel- 
ing of empowerment begins to manifest itself. 
Both awareness and action are important for 
building empowerment. It is the continuous 
linking of knowledge with action that sustains 
and strengthens the process of empowerment. 
A collective analysis of strength and wea- 
knesses of individual and group effort facili- 
tates the process of empowerment. 


5. Mobilization 


Where awareness is high and empowerment 
exists,, mobilization is crystallized through the 
process of participatory evaluation. There are 
innumerable instances in WDP to show how 
mobilization has taken place as information is 
shared, discussed, analysed and a common 
action is planned through various forums of 
evaluation 


6. Organization-building 


The process of organization-building is en- 
hanced by participatory evaluation because of 
its emphasis on a collective process. Since it 
deliberately brings people together for collec- 
tive analysis, a feeling of solidarity begins to 
develop and this facilitates formation of groups. 
Although WDP has helped in building solidar- 
ity amongst its various functionaries, forming 
women’s groups and organizing them in vil- 
lages is still a major challenge. 


7. Decentralized action 


Participatory evaluation has encouraged local 
initiative and decentralized action among 
functionaries at the village level. Rather than 
depending upon external help, the trend seems 
to be towards greater self-reliance in planning 
and carrying out evaluation on the basis of 
their own understanding. 


8. Changing role of outside evaluator 


Finally, the role of the outside evaluator has 
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‘undergone a change due to the process of par- 
ticipatory evaluation. Initially, this role was 
perceived to be that of a ‘facilitator’. Evalua- 
tion experience, however, has now shown that 
as ‘analysis’ and ‘reflection’ have become part 
of the WDP ethos, the outside evaluators have, 
in addition to being participant evaluators, 
have ‘insider-outsider’ roles to play in the par- 
ticipatory process of evaluation. 


Conclusion 


Ls The experience of 18 months shows 
that the pace of WDP , as compared to 
other government programmes, has 
been slow and cautious. It is still being 
consolidated in the six districts as per 
the original plan in 1984. This pace is a 
direct outcome of the ‘critical reflection’ 
which has become an invariable com- 
ponent in all the processes of WDP . 
The participants feel that the ‘gradual- 
ness’ of the implementation process is 
an essential strength of any develop- 
mental intervention at the rural level. 


2 In terms of significant material changes 
in the rural context, there is little to 
offer from the WDP experience. The 
poverty alleviation programmes run by 
the government have been subjected to 
a grass-root censorship through WDP 
in an attempt to render them more 
meaningful. This relates primarily to 
the preparation of beneficiary lists and 
IRDP disbursement of benefits. The 
role which the sathin-pracheta input 
has played is mostly by way of generat- 
ing village-level group reflections on 
the effectiveness as well as appropri- 
ateness of the training provided through 
TRYSEM and the procedure of disbur- 
sement of loans by the bank 
functionaries. 
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power relationships at the village level. 
No sensational turmoil has been stirred 
up which could be glorified as a mass 
movement led by women. There are 
only sporadic events giving a possible 
pattern which could, if strengthened, 
make women feel ‘more equal’ than 
before in their social contexts. 


4. As an experiment in group approach, it 
has had positive as well as negative sets 
of experiences. It has achieved a signifi- 
cant shift in women’s perspective on 
their problems. They have shown a 
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Qualitatively, there is no radical shift in 


movement away from attributing their 
suffering either to their ‘fate’ or to a 
personal shortcoming. Most of them 
can see it now as kinds of problems 
which are shared by other women also. 
This, in turn, has resulted in their expe- 
riencing a certain self-confidence and 
strength to solve their problems collec- 
tively. However, it has also resulted in 
breaking the ‘peace’ which is ensured by 
maintenance of the status quo. There 
have been problems with the village- 
heads who disapprove of women mov- 
ing away from the traditional role of 
being the ‘obedient’ members of the 
family. There have been panchayat dis- 
approvals by way of ‘fines’ levied on 
upper-caste women because they have 
been eating with scheduled-caste 
women. 

py A’s an experiment in minimizing, hier- 
archical relationships, WDP has got its 
greatest positive experiences. The pro- 
gramme has generated a personal con-’ 
fidence in almost all those associated 
with its activities. This has led to a 
common understanding of a world- 
view that recognizes that the rural 
women have to be treated with respect 
and dignity. Their ability to contribute 
in thought as well as action is as impor- 
tant as their urban counterparts. This 
recognition is an essential step in solic- 
iting their participation in the deve- 
lopmental processes. 


This attempt at capturing some of the main 
patterns as they are emerging in WDP ts 
neither comprehensive nor predictive. The 
future horizon is hazy. However, one fairly 
certain reading that can be made is that if the 
central values adhered to in this programme 
continue to be those of ‘participation’ and 
communication and if the  evaluation- 
monitoring system continues to be what it is 
now, there is a good chance that the rural 
women would move many steps ahead towards 
joining the larger processes of development. 
The beginnings have been made with a posi- 
tive spirit. It still needs to be seen how far the 
larger system gears up its protective mecha- 
nism and allows these small beginnings to 
grow into self-supporting processes. 

ANITA DIGHE, SHARADA JAIN, 
KAVITA SRIVASTAVA, KANCHAN MATHUR 


Further details about this Programme are contained in 
“Exploring Possibilities: A_ Review of the WUDP, 
Rajasthan,” published by the Institute of Development 
Studies, B-124 Mangal Marg, Jaipur-302 015 ‘ 


THE ANKUR EXPERIENCE 1978-1988 


New Delhi, India 


Introduction and background 


Ankur has a cumulative experience of 10 years 
of work in the field of non-formal education 
with illiterate and drop-out children; adult 
education and literacy with girls and women; 
community organization and income genera- 
tion; working with youth; and doing pro- 
grammes of development education and com- 
munity work with high school students in the 
forma! public school system. 


A brief historical survey 


Our organizational history can be broadly 
divided into three phases, and to some extent 
our process of evaluation corresponds roughly 
to these phases. 


Phase I : 1977-80 


Ankur began as a small, school based project, 
called the Abner Memorial Non-Formal Edu- 
cational Project and took its initial inspiration 
from the. nationwide National Adult Educa- 
tion and Non-Formal Education drive launched 
in 1977-78. 


As its name suggests, the objective was 
clear, and limited to imparting literacy skills to 
out-of-school children, utilizing school child- 
ren as student teachers, and also utilizing the 
slot in the school timetable set aside for 
Socially Useful & Productive Work (SUPW). 
About 30-60 students of classes nine and ten, 
supervised by a small staff of two teachers and 
two or three social workers, went out to teach 
the kids in the community—comprising a 
learner group of about an equal number. 
Organizational structure—in keeping with the 
formal school structure—formal, hierarchical, 
' top down. 


FIRST EVALUATION : MAY 1979 


Initiators 


The Project Management 


Methodologies 


Mainly traditional methods were employed— 
using observation schedules, questionnaires 
and some interviews. 


While there was some participation of the 
implementing staff in designing and adminis- 
tering schedules, by and large the student 
teachers were not involved. The learners were 
mainly assessed in terms of improved skills. 


Impact 


Although direct impact on the ‘beneficiaries’ 
was negligible, the overall reactions of the stu- 
dent teachers and their parents to the pro- 
gramme itself were favourable enough to pro- 
vide an impetus to the organizers to expand 
the scope of the Inter School Non-Formal 
Education Project from one public school to six 
well known schools in Delhi. The NFE pro- 
gramme has come to remain as an important 
choice offered in many schools under the 
SUPW scheme. 


It is interesting that this first evaluation did 
not directly comment on, nor did it provide any 
fresh impetus to the growth of community 
based programmes. 


However, the outreach into direct com- 
munity-based programmes with the children, 
and then with the women, occurred almost as a 
natural outcome of the interaction with the 
children and through them, with the needs and 
realities of the community. This initially took 
the form of a need for literacy classes for 
women—but. gradually developed into a wider 
programme as the real needs began to be per- 
ceived and articulated with greater clarity. 


Phase II : 1980-83 


The Abner Memorial Project expanded its 
activities much more specifically into the 
community, continuing to work through school 
students, but also reaching directly into the 
community to identify individuals willing and 
able to teach under the most gruelling of phys- 
ical and material conditions. 

—New thrusts, new programmes 

—Working with women 

—cCraft, sewing and income generation 

— Adult education and literacy 

—Taking up issues 

—Networking with other groups 

—Increased activism 

—Developing a perspective 

—Rapid expansion of staff 


—Expansion of centres 
—Applications for more funds 
—Development educational programme 


—Non-formal education 
—Many new communities 


SECOND EVALUATION : 1982-83 


Initiators 


The Project Management 


Methodology 


Much more participatory than the first evalua- 
tion. A considerable amount of time and ener- 
gies were spent designing the format and 
‘instruments and working out the steps of the 
evaluation with the manager, director and core 
staff of the project. 


Implementation 


Given the fact that neither community nor 
field workers had initiated the evaluatory pro- 
cess, it was mainly the full-time functionaries 
who participated effectively and played a 
greater role in filling-in forms and pushing the 
process of interviews, tabulations and so on. 
Although discussions and community feedback 
were planned, there were major problems in 
sticking to the framework, and so, many of the 
scheduled events did not take place at all. 


We were also dealing with multiple consti 
tuencies— 

—schools and their management 

—student,. teachers and their parents 

—learners 

—community-based teachers 


— difterent levels of project staff and 
management. 


These made the actual dynamics of the eva- 
luation more complex. 


Impact and outcome 


There were a number of pertinent observa- 
tions as to the impact of the programme, need 
for improving training, and upgrading skills of 
functionaries. 

However, the most significant outcome in 
the context of Ankur as it is today were as 
follows: 


i. A directive to project organizers to fur- 
ther explore and clarify the relationship 
between the formal and the non-formal 
systems of education in the country— 


especially with a view to influencing pol- 
icy at a national level. 
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ii. An unambiguous statement to the effect 
that in its present form, the inter-school 
project benefited the middle class 
school-goer much more substantially than 
it did the child in the community. 


iii.. That even our community programmes 
did not really make any significant impact 
on the community. 


As a result of this second and more partici- 
patory evaluation process, certain significant 
directional changes were made and are impor- 
tant in the context of Ankur’s present and 
future, namely : 


i. Lower priority to’ the inter-school NFE 
work—gradual process of pushing the 
public schools to take over the pro- 
gramme as an intrinsic part of their cur- 
riculum, and also to absorb the social 
worker/supervisor in charge of NFE 
onto the school staff. 


ii. Shift of emphasis to an issue-based,, 
development education programme 
designed to heighten awareness and 
questioning in school students and 
community. 


iii. Strengthening the community-based 
work. 


iv. Finally, the break from the Abner School 
and Baptist Mission—which emerged as 
constraining influences. 


-y. Setting up of an independent entity and 


society—namely Ankur. 


Phase III : 1983-86 


—Registering Ankur 

—Finding our feet, finding space 

—Continuing with existing staff and 
projects 

—Inductirlg new staff 

—Charting out new directions 

—Changing structures and styles of func- 
tioning 

—new concepts, new concerns and ques- 
tions 

—collective decision making—how? 

—hierarchy vs democracy—what does it 


mean? 

—equality—fine in theory, tough in 
practice 

—activism and _ politicization—new chal- 
lenges 


—reorganizing administrative structure 


—searching for clarity, redefining objec- 
tives, creating perspectives 

—internal dissensions 

—Sswinging to extremes 

—inexperienced leadership 

—near anarchy 

—shortage of funds 

—on the edge of collapse 

—the question of collaboration 

—the break-up of Ankur predicted by many, 


At the same time 


—work must go on 

—a sense of purpose 

—we shall overcome 

—new friends, new projects, new directions 

—linkages with several new groups 

—involvement in major current move- 
ments as never before like “working 
with Sikh widows after November 
1984” 

—need to evaluate as a necessary part of 
survival. 


THIRD EVALUATION : 1985-87 
This too can be divided into three phases : 


Phase I 


a. The internal situation prevailing within 
Ankur, and the rapidly deteriorating work 
environment provided the basic compulsion 
(or propulsion!) for a series of steps which, 
although never officiaily designated as such, 
were in fact the first phase of our third 
evaluation. 


The executive committee and others had 
.to do something, act fast to control matters; 
therefore two committees were set up, the 
one leading into the.other : 


b. i. The Finance Committee. 
ii. The Working Committee. 

It is significant to note that both these 
evolved in response to an internal logic and 
need, not because of outside interventions. 

NB: The selection process and composi- 
tion of the Working Committee itself was 
an interesting exercise in participatory 
functioning. Area teams and project teams 
made choices; vertical and _ horizontal 
representation was stressed. 


Every aspect of Ankur was under review 


—direction of work 
—programme priorities 


—geographic spread 

—staffing patterns 

—salaries 

—relationship with other NGOs, govern- 
ment programmes and departments 

—structure of organization 

—funding sources 

—1inter-personal relations 

—and much else. 


1. In short, the need for evaluation was felt 
at all levels within Ankur. 

2. We evolved, almost instinctively, the 
mechanisms with which to begin the 
process of evaluation. (NB: The expe- 
rience of the second evaluation undoubt- 
edly helped create this ability). 

3. No outsider was involved in the process 
at this stage. 


Creating our own guidelines for change and 
growth 


The Working Committee met over a period of 
many months and some 15 meetings—and 
came up with a series of recommendations. 
These were then shared with all levels of staff 
and the Executive Committee, as also, the 
community. Most recommendations were 
accepted, with some modifications, and incor- 
porated into the new project proposals and 
budgetary requirements submitted to our 
funders : 


Some outcomes 


—cutting down of geographic spread 

—induction of community-based workers 

—strengthening community organization 
skills 

—identifying permanent communities, as 
also 

—groups within communities with a 
greater permanence viz. women, youth 

—redefining development education pro- 
ject so as to give it a definite community 
dimension and create linkages with local 
municipal schools and communities 

—recommending phased withdrawal of 
parent organization, namely Ankur, from 
an area within a fixed time-frame 

—strengthening the concept of ‘team as 
being central to Ankur’s style of 
functioning 

—despite intensive efforts to restructure 
the entire salary structure, it was felt that 
in the final analysis the existing salary 
structure was as just and egalitarian as it 
was possible to be, and given the outside 
environment, suitable and fair 


Some factors leading to phase I 


i. Information about our internal crisis had 
naturally beén communicated to our chief 
funder. 

ii. Ankur also submitted a budgetary pro- 
posal for a further period of three years. 

iii, The funding agency nominated their 
local consultant (the late Vikas Bhat), to 
intervene and help facilitate a process of 
reflection and evaluation, prior to taking 
any decision on future funding. 

iv. Vikas Bhai played a major role in ena- 
bling the group, not only to survive 
financially, but, and more important, to 
retain a sense of confidence in ourselves. 

The primary motivation for this phase of 
the evaluation was most definitely from an 
outside source—in this case the funding agency. 

Possibly this anticipated a development 
from within the group to likewise initiate such 
a process as a logical follow-up to the outcorae 
of the working committee’s findings. 

Important to us was the fact that apart from 
projecting the need for such an evaluation, we 
were given total autonomy in setting out the 
terms of reference, the framework, the guide- 
lines and methodology and in the selection of 
an acceptable facilitator who could work with 
the group through the evaluatory process. 

The only form of pressure (if indeed it could 
be called that) was a gentle hint and insistence 
that we might like to choose a woman :n pre- 
ference to a manas our facilitator. Through 
many sessions it came home to us—again 
largely because of Vikas Bhai’s perceptiveness 
and sensitivity, tnat while Ankur never called 
itself a women’s organization, our image was 
that of a group composed primarily of women 
and deeply involved with women’s issues. It is 
perhaps for these reasons that the choice of a 
woman for this role was. appropriate. 

This fact takes on an additional significance 
in the light of later experience which has led us 
to reflect precisely on this question of gender 
in participatory evaluation. 


Some preliminary steps 


—Meetings with various levels of staff and 
the Executive Committee to talk about: 
Why evaluation? Why not? 


—Inviting suggestions and exploring suit- 


ability of various people for a role of 
facilitator insider/outsider, known /un- 
known, pros/cons. 

—Three or four names shortlisted, more 
meetings, and final selection after consi- 
dering time and other factors. 
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More steps 


—Working out a broad schedule for 
evaluation. 

—Individual and group discussions at all 
levels both within Ankur, and with some 
others such as Bina Agarwal, Rajesh 
Tandon, who had experience. 

—Arriving at a consensus as to metho- 
dology. 


Decision 


As far as possible we would make this eva- 
luation participatory, non-threatening and en- 
joyable. This would be an evaluation of pro- 
gramme and not of individual performance. 


Observations 


After some initial hesitation and reluctance— 
people began to feel comfortable with the » 
idea of this evaluation despite the disruption 
and demands on time. This was partially 
because of their involvement at every stage of 
the planning; and largely because of the per- 
sonality and temperament of the facilitator, 
who was easy-going, relaxed and fun. At the 
same time she commanded a respect because 
of her own professional approach and com- 
mitment. She did not impose or dominate in 
any way. Nor did she come across as threaten- 
ing or patronizing. Both these are important 
factors and came out of several informal eva- 
luations of the facilitator. 

Therefore does the choice of facilitator/ 
evaluator significantly affect the nature of the 
evaluatory process? And, if so, how? These 
become important questions to ponder. 


Some further steps 


i. Meetings with entire staff—full and part- 
timers. Lengthy discussions and expla- 
nations. 

ii. Why evaluation—whose need? Fund- 
ers? Ankur? The Community? Individual? 

iii. How—Evaluation? Instruments, criteria, 
interviews, schedule—all to be done by 
all of us. | 

iv. Therefore—understanding the nature of 
participatory techniques in evaluation, 
and how this process differed from tradi- 
tional modes and approaches to 
evaluation. 

v. Role of an outside facilitator—balancing 
the needs of objectivity and sensitivity. 

vi. Planning together; dates for meetings 
with whom? 
—The community 
—Learners 


Positive 


@ Existing conflicts and tensions 
were highlighted objectively. This 
helped resolve them. 


e Programme weakness and non- 
functioning were openly 
acknowledged. 


@ This led to hard but necessary 
decisions like closing down of 
some projects, revamping others. 


@ Resulted in some members leav- 
ing the organization. 


® Reinforced and _ sttengthened 
decision making processes by 
those authorized to do so. — 


—Part-time teachers 
—Full-time staff/coordinator 
—Area teams 
—Project teams 
—Directors 
—Executive Committee 
—Concerned individuals 
vii. Working out the actual questionnaires, 
interview schedules and criteria of assess- 
ment with each appropriate group! 


This process alone took many meetings, 
many hours! 


Phase III : Some causative factors 


1. Thecomposite outcomes of the entire 
process comprising Working Com- 
mittee Report;, Evaluation Report; 
Executive Committee's responses, an 
overview of all Ankur’s projects; and a 
new budget proposal; all these: were 
finally submitted to the primary fund- 
ers around October 1986. 

2. The funding agency, after an indepth 
study and analysis of the entire pro- 
cess, approved the proposal in 1987. 

4 There had been a gradual easing of 
tension and strain and insecurity at all 
levels in Arikur over this period. A 


THE PROCESS—SOME OUTCOMES 


Negative 


@ Delay in submission in final re- 
port caused avoidable prolonging 
of tension in Ankur. 

@ Submission of the evaluation 
report in English and lack of 
Hindi translation at the right 
time led to unnecessary misunder- 
standings and misinterpretations, 
genuine or deliberate. 


« 
@ - Boththese above factors accentua- 
ted the extraordinary level and 
nature of adverse often ill-in- 


formed criticism, from groups 
and individuals peripherally or 
otherwise linked with Ankur's 
work | 


new mood of gearing up to more posi- 
tive and efficient and productive work 
‘was evident all round. 

4. All these cumulative experiences 
seemed to have led to a more critical, 
self-reflective mood at all levels too. 

5. The involvement of various levels of 
personnel in assessing and where pos- 
sible, revising part of the salary scales 
upwards, had also called forth and 
indicated the need for not only on- 
going evaluation of programme, but a 
more critical evaluation of individual 
performance. 


A schedule of early events 


December 1986/January 1987 


A series of informal discussions with individu- 
als and groups within Ankur, as well as earlier 
evaluators. 


February 3, 1987 


Performance evaluation discussed formally at 
staff meeting. After sharing reactions and 
some reservations, a suggestion made and 


accepted to call in Dr Rajesh Tandon from 
PRIA to facilitate a day-long discussion on 
“Why individual evaluation? Can this be a 


non-threatening and participatory exercise?” 
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how 


Even this step of inviting someone to facili- 
tate a discussion on evaluation did give rise to 
misgivings among some as to the inherent 
biases of ‘experts’ and ‘outsiders’ per se. 


February 13, 1987 


Day-long event—indepth discussion on sev- 
eral aspects of evaluation—based on an initial 
presentation by Dr Rajesh Tandon of PRIA, 
wherein several issues were raised; questions 
and doubts answered; fears allayed. General 
agreement on decision to go ahead with per- 
formance evaluation. 


Small groups set up to work out the final 
format of questionnaires based on framework 
of criteria and other details as discussed and 
agreed at the large group session. 


Sharing what happened 


The agony and the ecstasy 


—Never before had we been through a 
process where we told each other things 
that we felt about each other, so openly, 
in a group, face to face... it was hard at 
first, but it felt good... 

—We remember the agony of deciding 
what score to give each one... If I marked 
my colleague low on openness, would she 
do the same to me? How would it affect 
my final grading? 

—Even more tough—to find rational, 
cogent and well founded reasons to sup- 
port our marking in front of a whole 
group. We could no longer take shelter in 
remaining anonymous... felt insecure 
too... 

—Sharing the anguish of a senior woman 
like Shanta Devi—one of our oldest tea- 
chers—today a senior co-ordinator— 
gritty and hardworking; a first-rate field 
worker, but unable to cope with the 
world of abstractions and concepts. Each 
time a low score was given to Shanta 
Devi by any of us, her colleagues, her 
friends, it was a blow to her whole being, 
her self-esteem, and her assessment of 
herself. 

—Too many questions and dilemmas. Were 
we utterly cruel and heartless? Was all 
this really necessary? 

—Can we dare to tell our ‘director did? 
what we don’t like about her and not fear 
for our jobs? 


Slowly, very slowly, people began to see 
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what this kind ot evaluation was all about... 


Began to enjoy it... 


—It was such a good feeling to be able to 
point out each other’s weaknesses and 
faults without feeling a sense of guilt... — 

—We all cried together with Shanta Devi; — 
_ but oh! the exhilaration when she finally 
began to understand that by marking her _ 
low on openness it did not mean that we 
were biased against her... 

—It took time, but we all learned to make | 
statements about each other only after 
serious reflection and analysis. 

—Till. now we had tended to talk behind 
each other’s backs—it was so much bet- 
ter to say it face to face... 

—We learnt to trust one another, or at least 
we realized that in order to work as a 
team, it was necessary to learn to trust... 

—We were also helped to see that when 
scores were allocated on so many levels— 
from self, to peer group, to community 
and to directors, and then averages taken, 
‘this was about as objective a scoring 
method as it was possible to get... 

—Most important—When, by mutual 
agreement halfway through the individ- 
ual evaluation we decided that we would 
not link: the results of this evaluation 
with salaries and increments—we began 
to see much more clearly that this was 
indeed our evaluation—we owned it as it 
were—and therefore it was not to be 
feared. 


The doubts and fears—the blocks and 
problems—the questions 


—‘People’ say that this will be used against 
us—to throw us out. What about our 
~ jobs? 


—Other groups say they only do verbal 
evaluations—they don’t have to go around 
filling up forms. Why did we have a writ- 
ten and a verbal evaluation. What does it 
prove? Which one is better? 


—We belong to a joint project—and this 
means we are being evaluated twice— 
once by them and a second time now. 
That is nor fair! 


—Too many meetings—is it really neces- 
sary to go through all this for an evalua- 
tion? The community work, our.classes 
are suffering? How can we explain it to 
them? 


—Despite all our efforts, it has not really 
been possible to.convince the commun- 
ity, nor to really get a proper evaluation 
from them. They continue to’ say nice 
things—but we don’t really know what 
they think about us. Maybe we need to 
work with them to be more critical? 


—Can people really be honest in evaluation? 


—Should not we be doing evaluations 
continuously—and not just once or twice? 


Some lessons learnt—insights gained 


1. It’s interesting that evaluation can be of 
so many different kinds—stated and un- 
stated, overt and covert. In our entire 
evaluation—the formal. evaluation— 
there has been no mention of the prob- 
lems that we were facing with one of our 
collaborative projects. And yet, over the 
whole period of 1987, we held about a 
dozen meetings, many small group in- 
formal sessions during which we talked, 
we argued, we clarified, and finally decided 
that it would be better to work separ- 
ately. Wasn't that also a process of 
evaluation? 


2. The overall teeling today, is that the 
entire exercise has left us feeling positive 
and good about ourselves and our work. 
We are not feeling bruised or left with 
damaged self images. And if we can look 
back and laugh at the tensions and set- 
backs during the process of evaluation, it 
must mean that everyone was genuinely 
involved at every stage—and therefore 
it was participatory. 

3.  Inourown ways, we have all individuaily 
and collectively realized that PE is not a 
static, time-bound exercise; rather we are 
seeing it as a continuum—a dynamic 
process where each phase is linked by an 

_ internal logic to what went before, and in 
turn creates the need for the subsequent 
phase. 

4, “while working on this very presenta- 
tion—we have identified some of the 
steps we need to take so that what we 
share here will be further communicated 
to all levels of those who work to- 
gether—and again, in a medium that 
does not restrict itself to the written 
word alone. 


1. The ‘gender factor’ in evaluation 


Do women use the tool of evaluation in a 
different way from men? How does this differ- 
ence manifest itself? What is its impact on 
efficiency? On objectivity? On productivity? 

The question is being raised here because, as 
mentioned earlier, our attention was drawn to 
this at the very beginning, in the choice of a 
woman as facilitator in Phase II of the Third 
Evaluation. It has been reinforced by our most 
recent experience during the period of indi- 
vidual evaluations—where we constantly 
found ourselves giving the time and the atten- 
tion to so many of the special problems that 
women face in the area of work and perfor- 
mance; their daily dilemmas of coping with the 
dual responsibilities of home, children, 
demanding husbands or mothers-in-law, as 
well as the strenuous and occasionally uncom- 
promising demands of the workplace. And yet, 
we believe that by and large this sensitivity to 
the personal factor, did not substantially affect 
the objectivity of evaluation. 


2. Participatory evaluation as a human 
process 


Quite instinctively as a group we found our- 
selves responding more gently in situations 
where we were. all aware that the one being 
evaluated had been through some personal 
trauma. 

Does this only happen in Participatory Eva- 
luation? Can this find a place in other forms of 
evaluation? 


3. Humanizing the other 

Leading from’ the above two observations—a 
third question— What if any attempts have 
been made for traditional and participatory 
modes of evaluation to interact, and if possible 
for Participatory Evaluation to help humanize 
the other? 


4, The real strength 

Finally—we have become aware of something 
that we can only term an inbuilt dialectic pro- 
cess in Participatory Evaluation and which is 
its real strength. It pushes each individual, and 
through them, the group—to develop the crit- 
ical and analytical capacity. This can only 
enrich both individual and collective function- 
ing leading ultimately to qualitatively better 
action. 


In conclusion 


> 
This entire process that. we at Ankur have 
gone through has left us with two overwhelm- 
ing feelings about Participatory Evaluation. 
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ANKUR, ALARIPPU: LALITA, LAKSHMI, MARTHA, 
JAYA... AND ALL THE OTHERS WITHOUT WHOM THIS 
PROCESS, THESE REFLECTIONS WOULD NOT HAVE 
BEEN POSSIBLE. a 


RURAL HEALTH PROJECT 


Maharashtra, India 


I. History of the organization 


A rural health project which started in 1977, 
now covers 22 villages (total population 
38,000) in two tehsils of a progressive district 
in Maharashtra. It is known as one of the 
pioneering work in introducing Primary 
Health Care approach. The very objective of 
this project is “to develop a system of 
comprehensive health care through primary 
health care approach with emphasis on com- 
munity participation’. 


The village people are selected and trained 
to work as voluntary health workers to act as 
an interface between the community and the 
health infrastructure through health care 
institutions such as sub-centres, health centres, 
rural hospital and a full-fledged hospital at 
district level. The entire project involves 40 
community health volunteers, 15 multi-pur- 
pose workers (employed by zila parishads and 
deputed to this project), 10 supervisors and 
administrative staff and three directors at the 
top as policy makers. 


The supervisors and administrative heads 
monitor the functioning closely by explaining 
the work schedule and taking feedback about 
completion of the target, through a monthly 
meeting. The patients are being identified, 
_ diagnosed and treated or if found complicated 
referred to hospital and specialists, medicines 
are being distributed, children are:being regu- 
larly immunized, anti-natal cases enrolled and 
followed up, family planning methods are 
propagated and operations are conducted. And 
all these cases are increasing in number every 
year. Various studies have also proved that 
there is a significant increase in the health 
status of village people in the project area, 
evidenced by fall in crude death rate, infant 
mortality rate as well as improvement in 
mother and child health and family planning 
services including immunization. 


The project nas been in operation for over a 
decade now, but the above assessment of the 
functioning has been done only by the policy- 
makers and the administrative heads and that 
also of only health measures. The social, educa- 
tional aspects of the project are not assessed. 
The community and the community health 
volunteers are not consulted to measure the 
impact of the functioning of the project. 
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Following are the three crucial objectives of 
the project and of primary health care 
approach. Being concerned about these and 
realizing the need of achieving these the 
administrators, the policy-makers of the pro- 
ject thought of taking opinion of the commun- 
ity about it. These objectives are: 


a. Participation of community in delivery 
of health services. 


b. Increase in health awareness of village 
people. 


c. Replicability of the model used in this 
project elsewhere. 


These three objectives are discussed, ana- 
lysed and stressed on further planning and 
extension of the project, only at policy and 
administrative level, and are found to pe 
incomplete. On this background, the idea of 
Participatory Evaluation proposed by Associa- 
tion for Applied Social Research was found 
appropriate and was operationalized at once. 


Thus there was no history of participatory 
evaluation, nor of any practice or platform to 
call for collective thinking and review of the 
project. In fact, the concept of evaluation was 
“data-collection on various health indices”. 


II. Objectives of the participatory 
evaluation 


Following specific objectives of evaluation 
were defined in consultation with the policy 
makers of the project, which are based on their 
perception and vision of the project, and out- 
lined their expectations from the evaluation. 


1. To find out whether the community has 
acquired adequate health awareness by 
the intervention of the project staff. Do 
they find this awareness useful? What 
other information and knowledge 1s 
gained by-the community from the servi- 
ces of the project and how are they bene- 
fited by that? 


2. To identify the factors which motivate 


the community to avail of and accept the 
health and other services of the project. 


3.  Tostudy how far the community is moti- 
vated, trained, prepared and given scope 
to participate in the functioning of the 
rural health project. What are the obsta- 


cles in their participation? What are the 
drawbacks and strengths of their active 
participation in the project? 

4. To make recommendations to streng- 


then effective community participation 
in the project. 


Additional objectives for evaluation were 
expected to emerge from discussions with field 
functionaries—CHVs and MPWs, and the 
community, while involving their participa- 
tion in the evaluation process. 


Ill. Methodology and process of partici- 
patory evaluation, as it was initially 
planned 


The policy makers of the project had stressed 
the point that they wanted to know the com- 
munity’s perception about the project. Hence 
the participation of community and €HVs is 


underlined in each phase of +the evaluation. 


process. 


Phase I : Preparation 


It was proposed that a survey, a comprehen- ° 


sive examination of people’s attitude towards 
Rural Health Project by gathering first-hand 
data is conducted initially to acquaint ourselves 
(external evaluation assistants) with the func- 
tioning and background of the project. This 
also should be supplemented by secondary data 
of the studies already conducted. The focus of 
the survey was to cover following questions: 


1. How does the community perceive the 
role of CHVs, MPWs, ANMs, Medical 
Officers, and of the apex institutions? 


2. | What problems are perceived by the com- 
munity in general and beneficiary of the 
project in specific, in the functioning of 
the project? 

3. What were and are the expectations of 
the community from the project? How 
far these are met? 

4. What are the commfunity’s suggestions 
for improvement in the PHC approach 
and the project functioning and what 
role they are prepared to play in it? 


The objectives of this phase were set as to 
a. establish rapport with functionaries and 
community; 


b. identify and define the problems (if any) 
perceived by the community in the 


project, 


c. involve the community in participatory 
evaluation; 


d. select key individuals from the com- 
munity to focus core evaluation team. 


This phase was ‘expected to be of three 
months. 


Phase II : Intervention 


Having identified issues for evaluation and 
individuals who will participate in it, the actual 
evaluation process was to begin in this phase. 
The -methods and focus of evaluation in this 
phase were to be determined based on the 
experience and outcome of Phase I. Broadly 
they were jotted down as: 


a. tohighlight the strengths of this project. 


b. to find out the root causes of the pro- 
blems/obstacles/difficulties perceived by 
the community and project staff in three 
aspects (namely community and project 
participatioh, health awareness, and repli- 
cability of the project) which form the 
focus of the evaluation; 


c. to list the suggestions made by all those 
involved in evaluation to achieve better 
results in the above mentioned three 
aspects gf the project; 


d. to critically evaluate attempts to bring 
about improvements by the individuals 
in the community and project personnel; 
and to prepared action plans to over- 
come the obstacles and improve the 
effectiveness of the project. 


These objectives would have been fulfilled by 
either 


—conducting indepth interviews of the 
selected individuals in the community 
and in the project personnel; or/and 


— organizing group discussions of the indi- 
viduals in the community and in the pro- 
ject personnel both separately and com- 
bined; or/and 


— organizing workshops, overnight camps 
of key individuals in the community, 
other voluntary groups such as Mahila 
Mandals, other social workers in the 
area, as well as project personnel and 
Zila Parishad staff. 


It was assumed that some other suitable 
methods may have to be selected to cover the 
above listed objectives depending on the 
response choice of village community. 


This phase was expected to continue for a 
minimum of six months. 
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Phase III : Report writing 


Though a day-to-day process record was to be 
maintained and periodical reports to be pre- 
pared for discussions with coordinators of the 
project; the final report of this participatory 
evaluation was to be written both in English 
for policy-makers and in Marathi for the 
community. 


IV. Methodology and process of par- 
ticipatory evaluation as it emerged 


The elaborate design described in earlier pages 
did not materialize all that smoothly. Follow- 
ing steps were planned and taken to invite and 
respect the participation of community and 
other functionaries, who formed the evalua- 
tion team: 


1. First of all, the team of external individ-. 


uals participating in evaluation (from 
Association for Applied Social Research) 
visited all the villages till they got acqu- 
ainted with all and the functionaries 
became aware of the background of the 
project. 


2. Secondly, in group discussions that fol- 
lowed the visits (in each village and 
combined for all the villages together), it 
was confirmed that the need for evalua- 
tion is strongly felt, before starting actual 
process of evaluation. 


3. Thirdly, a core team of evaluators was 
formed to take basic decisions and collect 
data like what should be the focus of 
evaluation, methods of evaluations, in 
which villages intensive evaluation should 
take place and why, etc. 


4, Fourthly, all the members of the evalua- 
tion team were assured that the confi- 
dentiality of their information would be 
kept and still the evaluation would be 
seriously brought to the attention of 
policy-makers for action. 


5. Though it may sound simplistic, in the 
context of this particular setting follow- 
ing action implications also proved to be 
useful in creating atmosphere and 
emerged participation, like 


a. keeping the time given for them in 
the meeting which was unlike the 
other meetings of the project; 


b. reaching the remote areas despite 
the unavailability of transport facil- 
ity, whenever it was jointly decided; 
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c. deciding time, date and venue accord- 
ing to the convenience of commun- 
ity and community health volunteers; 


d. sitting along with them on the floor 
in a circle; 


e. using local examples for illustrations 
etc. 


All these actions did result in spontaneous 
response, initiative in organizing andattend- 
ing group discussions, uninhibited analysis and 
even expression of willingness to face negative 
consequences if it meant action for solving the 
problems in the project. The rapport got estab- 
lished so fast that after field visits the com- 
munity and functionaries both feit that Phase I 
as it was planned can be combined with Phase 
II. In fact after the field visits it was only 
through intensive weeklong discussions that 
all the objectives of evaluation were covered 
(except about the replicability) and there was 
no need of interviews or survey. The flexibility 
of approach itself gave scope to participation 
and they did feel having control over the 
process. 


The entire process of evaluation took only 
half the scheduled time. 


V. Outcome of evaluation 


Since the project is still incomplete for the 
reason of confidentiality it is not possible to 
give the analysis and recommendations which 
emerged out of the evaluation in detail in this 
paper. But in brief. 


1. A list of problems at 

community level 

community health volunteers level 
MPW level 

supervisory level and 

policy level was prepared. 


see ei op ee 


2. The root causes behind these problems 
and their interconnections were traced. 


3. A strong need for change in the project 
implementation and in attitudes and 
approaches of administrative and super- | 
visory level personnel was expressed. 


4. And before proceeding any further the 
CHVs and MPWs demanded immediate 
action on the problems expressed by 
them. 


When the policy-makers and administrative 
head were informed about the above need and 
demand, they did not accept the evaluation and 
have at present suspended evaluation project. 


The community and CHVs nevertheless are 
interested in following up the evaluation. 


Some specific follow-up plans are also indi- 
cated to the policy-makers. 


VI. Impact of evaluation 


As a result of evaluation process as expressed 
by them 


a. The community health volunteers 
realized the importance of their role in 
the infrastructure. 


b. They felt confident of analysing and ex- 
pressing their problems anda meaning 
in doing this. 


c. Atone point there was also a hope raised 
in them that the problems wouid be 
solved, which—due to delay in response 
from the administrators and foreclosure 
of the evaluation project—they are los- 
ing now. 


d. They are thinking of alternative projects 
to sustain their roles as voluntary workers 
if it means being thrown out by this pro- 
ject or resigning from it themselves. 


e. Thesupervisors and administrators have 
begun to understand the interconnection 
of the problems and its relation with 
their role in a new perspective, that is of 
those for whom the project began 1.e. the 
community, and of those who work in 
the field. 


f. | Thesupervisors and administrators have 
also tried to suppress the evaluation, 
when they realized the potential threat 
to the very existence of the project. 


As a result of a negative feeling collectively 
expressed by the field functionaries, the dis- 
comforts felt and a realization of potential to 
work in this field, they have already begun to 
think in terms of independently approaching 
funders for similar health projects. The leader- 
ship has also emerged. And at present there is 
a possibility of organization-building. 


VII. Issues in participatory evaluation 


As an éxternal agent in the participatory pro- 
cess, one has felt the limitations as well as 
strengths of this role. But whenever the con- 
flicts between the levels of. hierarchy in the 
organization come to the surface of evaluation 
the external agent is puzzled. If the adminis- 
trators themselves have invited the external 


agent and they are being exposed in the pro- 
cess by field functionaries and the community, 
the situation becomes awkward for the exter- 
nal agent. After all the evaluation is also a 
paid-for activity by them and if they withdraw 
infrastructural ‘support (like transport, meet- 
ing place etc.), the external agent finds it diffi- 
cult to sustain only on the goodwill for people 
and development. 


SEEMA KHOT 
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RURAL DEVELOPMENT ORGANIZATION 


Madhya Pradesh, India 


Rural Development Organization (RDO) is one 
of those organizations which began their work 
in the mid seventies with a focus on conscienti- 
zation and organization of socially and econom- 
ically deprived and exploited sections of the 
society. Major thrust of their initial work was 
“Voice to Voiceless” and "Power to the Power- 
less”. RDO started its work in 1975 through one 
field centre in five villages in one of the most 
backward districts of Madhya Pradesh. In 1979- 
80, it established two other field centres and in 
1984, it once again expanded into new areas. 


In fact 1975-1979, 1979-1984 and 1985 on- 
wards are three distinct phases in the lifecycle 
of RDO. During each of these phases, several 
internal and external evaluations of RDO took 
place, and a considerable amount of written 
material on its strategy, philosophy and pro- 
grammes are available. It has come to be recog- 
nized in India and outside as a trendsetter organ- 
ization for operationalizing conscientization 
and organization model. With a successful cam- 
paign of political education and building up of 
people's organization in the field centres’ areas 
during the first two phases (1e., 1975-1984) it 
has not only expanded its work in 350 villages, 
but has also added some socio-economic and 
health programmes alongwith people's organt- 
zation. Introduction of socio-economic progra- 
mmes in the third phase, has started having 
some repercussions on the organizational func- 
tioning and work culture. 


At this juncture in a meeting of RDO repre- 
sentatives and representetives of all major 
funders of RDO, it was felt that if through a 
process of participatory evaluation a perspec- 
tive and organizational strategy for the next 10 
years could be evolved it would help in doing 
programme planning and thus facilitate com- 
posite funding for 10 years. In this meeting with 
mutual agreement between RDO and its donors 
on the above need, an agreement was also 
reached on the names of three potential agen- 


cies in India, one of which could play a facilitator» 


role for the proposed process of participatory 
evaluation. 
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By the time follow-up actions were taken on 
the agreements of this meeting and potential | 
agencies were contacted, RDO had faced some 
problem from the government and restrictions 
were put on RDO on receiving funds, its records 
were sealed by government officials and enquir- 
ies were set up. It had intensified the already 
brewing tension in the organization caused by 
the strains on organizational functioning and 
conflicts of work culture in its third phase, but 
due to the uncertainty created by new crisis most 
of the people in the organization were keeping 
their fingers crossed, when the first planning 
meeting was organized between the interven- 
ing agency and the senior staff of RDO. The 
planning meeting was attended by a team from 
intervening agency, representatives of other 
partners of RDO, a tentative representative of 
donors and senior staff of RDO. Apart from 
setting-up objectives of the proposed evaluation 
and developing the frame of reference, a detailed 
time schedule was also prepared. The objectives 
of evaluation derived in this meeting are as 
follows :— | 


i. Toassess the model followed by RDO as a 
mechanism for people's empowerment; 


ii. To assess the relevance of different pro- 
grammes in this model; and 


iii. To assess the appropriateness of the 
organizational form of RDO. 


The process 


With the above objectives in mind, it was felt 
that evaluation should focus on the experien- 
ces of the model followed by RDO as a model of 
development (i.e., conscientization organiza- 
tion model) and examine RDO’s programme 
and organizational structures in that light. 
Thus, the essential thrust of the evaluation was 
not on assessing impact on beneficiaries, but 
on the viability and applicability of the model 
and its appropriate supportive structures. 


At the end of the meeting, some questions 
were raised regarding the suitability of timing 


for the proposed evaluation process in the 
light of the crisis RDO was facing. After careful 
thinking based on the assessment of RDO staff 
present at the meeting, it was decided that the 
evaluation team will visit selected field areas 
and discuss the basic tenets of participatory 
evaluation with responsible area staff members 
and carry the evaluation process forward, tak- 
ing stock of their reactions. 


As per this understanding the: evaluation 
teams visited the areas and held meetings with 
area staff. These meetings were quite useful in 
terms of creating confidence among the staff 
members to candidly assess the need of partic- 
ipatory evaluation and the appropriateness of 
timing in conducting the same. In the course of 
deliberations, it became clear that staff was 
unmoved by the present crisis and was ready to 
join in the evaluation if their participation is 
ensured till the finalization of the report and 
planning of follow-up actions. 


Some of the staff considered the proposed 
evaluation as an exercise to merely satisfy the 
formalities of the donors and their organiza- 
tions. They said that this consideration was 
based on their past experiences when they 
neither saw the reports, nor saw any actions 
flowing out, of earlier evaluations. However, 
with repeated assurances of discussing the 
draft report with them before finalizing it, the 
evaluation team could win-over their confi- 
dence in some places; and in other places, some 
staff still held their doubts and wanted to 
assess the credibility of the evaluation team. 
This visit of the evaluation team was thus quite 
fruitful in creating an environment for carry- 
ing out the evaluation process. 


After discussing these impressions with the 
Executive Director and the Acting Executive 
Director, it was decided to go ahead with the 
process. The dates for the next visit of differ- 
ent members of the evaluation team were 
finalized and informed to the Acting Executive 
Director, who was made incharge of coordinat- 
ing the work with the evaluation team. 


During the second visit, as per schedule the 
scenario had drastically changed. All the per- 
sons from RDO concerned with the evaluation 
process were not available. One of the evalua- 
tion team member proceeded to the area as per 
schedule on her own, and interviewed some of 
the staff. Another member reached afterwards 
but could not break much ground because of 
the prevailing state of confusion due to legal 
crisis still facing RDO, coupled with flood relief 


task and some dynamics in organization un- 
known to the evaluation team till then. Per- 
haps, the dynamics was escalating faster due to 
prevailing uncertainty in the organization. 
After prolonged discussion with the Acting 
Director the next visits were planned and an 
assurance was given for all arrangements to be 
made for extensive field programme of the 
evaluation team. 


In the meantime the resource person 
involved in facilitating the evaluation process 
for social forestry programme had fixed a 
meeting of concerned staff in order to initiate 
the process. The internal conflicts and dynam- 
ics among senior RDO staff somehow stalled 
the process from moving further and the con- 
cerned resource person was forced to withdraw 
from the evaluation process. 


The interesting point at this juncture was 
that most of the responsible people in RDO 
were neither ready to take the stand to suspend 
the process of evaluation nor were they feeling 
comfortable in its progressing since they were 
not able to assess the possible outcomes of the 
exercise and its impact. With everybody over- 
tly accepting the importance of participatory 
evaluation exercise, promising all cooperation 
in future activity and apologetically regretting 
futility of the last two visits, it became a catch- 
22 situation for the team of the interventing 
agency. The agency could neither back out 
from the exercise on its own initiative nor do 
anything to expedite the process. “Wait and 
watch” was the only line of action left to deal 
with the situation. 


In order to keep the record straight, the . 
Acting Director of RDO was reminded of the 
coming visit of the team along with his assu- 
rances and two formats seeking some infor- 
mation. 


Strains on the process 


On the arrival of evaluation team, for scheduled 
visit, to its utter surprise the team was told that 
the process of evaluation was suspended at the 
instance of the donors as they want the process 
to continue only in the presence of their 
representatives. No information to this effect 
was sent to the interventing agency. On making 
queries, it was revealed that the decision to 
suspend the evaluation was made a week 
before this visit of the team, but for reasons 
unknown till today, it was not communicated 
to the facilitating agency. 
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This belated reaction of the representatives 
of donors on the issue of frame of reference 
and on the participation of donors further con- 
founded the already prevailing ajr of mistrust 
and confusion within RDO regarding this eva- 
luation. In fact, this intervention leading to the 
postponement of the evaluation process, had 
severely affected the basic fabric of participa- 
tion based on mutual respect and trust among 
the concerned parties. The displeasure of the 
facilitating agency was duly communicated to 
all concerned parties. 


Revelation of agendas 


After a lapse of two months a meeting was 
organized to sort out the misunderstanding 
among the concerned parties. But the meeting 
turned out to be an exercise of revealing the 
purposes of evaluation which were not made 
explicit till then. The issue of the appropriate- 
ness of time being conducive for evaluation of 
RDO was not raised by anyone but only the issue 
of donors’ participation with the evaluation 
team during field visits for data-collection was 
raised. The arguments given in support of the 
donors’ participation in the whole process of 
evaluation implied that if evaluation is done in 
their absence, it would not be worthy enough to 
convince the Board of Directors of donor organ- 
ization*and the philanthropic organizations 
behind them. This problem of convincing the 
Board of Directors of donors and philanthro- 
pic organizations behind them was neither 
mentioned as one of the objectives of the pro- 
posed evaluation during the consultation nor 
was communicated prior to this meeting. 


The facilitating agency was taken by surprise 
on this matter and took a stand in this meeting 
that the involvement of the donor and philan- 
thropic organizations 1n evaluation exercise at 
this stage and in the prevailing situation in RDO 
can be accepted till the level of headquarters and 
at the stage of planning and finalizing the pro- 
cess only. The facilitating agency made it clear in 
this meeting that it had agreed to take up this 
task because the objectives of evaluation implied 
assisting RDO to take stock of its past experien- 
ces and to plan for the future and because of the 
mutual trust reposed in the agency by RDO and 
donor organizations. Opening up of new fronti- 
ers at this late stage had belied both the pre- 
mises of the involvement of facilitating agency. 
At the same time the facilitating agency was not 
interested in withdrawing the helping hand toa 
fraternal organization when it was under attack 
from the government. This dilemma was finally 


38 


resolved by a compromise. It was decided in the 
meeting that the donors will send their team 
separately, which will be joined by some repre- 
sentatives of philanthropic organizations and 
some others. The findings will be shared with 
the facilitating agency. Somehow the deadlock 
was removed through this compromise. 


New phase 


Sensing the problems caused by the six month 
long postponement, and the developments 
within the organization caused by external 
pressures and interal dynamics, the facilitating 
team decided to prepare questionnaires for dif- 
ferent levels of staff, village council members 
and formats for programme and village coun- 
cil profiles. The team had also prepared guide- 
lines for indepth study of selected village coun- 
cils. In effect the team had changed its method 
of collecting information from group discus- 
sion; collective and individual reflection (which 
were used in first phase of evaluation) to rela- 
tively more structured methods of information 
collection. 


But when the team reached RDO, it was con- 
fronted with a new situation once again. In the 
briefing from the Executive Director, the team 
was told to stretch the period of evaluation since 
the retrenchment of animators/teachers had 
created new tension in the organization. The 
team was told that teachers were given compen- 
sation according to rule and asked to leave and as 
a result of dissatisfaction caused by such an 
action they were spreading rumours among the 
people. The Executive Director felt that 1n such 
a situation, the information the team collected 
might be coloured. 


At this stage the team explained to him that 
during this visit we had planned to concentrate 
on collecting information from the staff only 
and would not be holding group discussions with 
council members and other villagers. With this 
understanding, we proceeded to the field. Dur- 
ing the teams’ encounter with the staff, with 
whom the team had developed good rapport 
initially, the team was asked lots. of questions 
regarding the validity of the present process of 
evaluation in the context of happenings of the 
last few months. They were also doubtful about 
the locus stand of the team on the basis of their 
observations during the visit of donors’ team. 
They put forward straight questions to the team 
by asking about assurance of implementation of 
recommendations made by this evaluation, even 
if they cooperated with it. This was quite a 


learning experience for the team in terms of 
understanding the limitations of the evaluation. 


The deliberations with the staff members 
revealed that lots of actions were being initiated 
by those managing RDO regarding which the 
facilitating team had been kept in dark. Such 
- actions on the part of the management of RDO 
had changed the environment in the organiza- 
tione There was polarization around the issue of 
termination of teachers and reorganization pro- 
posed by the management. Most of the staff 
were only interested in discussing these issues 
and proposed a general meeting for that purpose. 


The state of the RDO at this stage was charac- 
terized by serious internal .conflicts among 
senior staff and Executive Director, extreme 
mutual mistrust, strategizing and counter-stra- 
tegizing between warring parties. - 


Hence in this trip as well, the team could 
carry out its scheduled work for four days only 
while rest of the five days the team simply 
observed the highly charged activity around the 
proposed meeting. Though this forced observa- 
tion exercise was quite a learning experience for 
the team in terms of developing insights into 
the internal dynamics and decision making pro- 
cess of the organization, and it was an important 
input to the evaluation exercise, but as a conse- 
quence, the team’s renewed attempts for syste- 
matic data-collection became a casuality once 
again. 


At this stage, the evaluation team felt a bit 
frustrated and doubtful about the intentions of 
the concerned parties regarding completion of 
this exercise. The team raised the issue with 
concerned persons and requested them to raise 
the issue in the forthcoming RDO governing 
body meeting and let the facilitating agency 
know the stand of the organization at the earli- 
est. The team proposed to hold a meeting of all 
senior and important staff of RDO to discuss the 
present state of affairs. The team felt that such a 
meeting, apart from being an exercise to pro- 
vide closure to the evaluation process, might 
also help in overcoming the state of stalemate 
faced by the organization then and temporarily 
manage the conflicts. 


On getting a green signal from them, such a 
meeting was finally held which took some cfiti- 
cal decisions to phase out the then crisis the 
organization was faced with. 


The above description gives an idea of stresses 
and strains which had aneffect on realizing the 
expected levels of participation of staff in the 


evaluation process and the originally planned 
expected objectives of evaluation. This account 
of events also brings out the limitations of the 
outside facilitating agency in sustaining’a partic- 
ipatory evaluation process. 


Based on the information collected in this 
process a detailed report was prepared and 
presented at a meeting, which was attended by 
40 persons including RDO governing body 
members, senior and important staff of RDO’ 
and evaluation team members. Summary obser- 
vations of that report are given below: 


Overall observations 


1. According to the original vision of RDO, it 
was expected to distinguish between the 
organization committed to the promotion 
of people’s movement and the people's 
organization created through its efforts. 
Such a conceptualization is visible in the 
initial documents, but it seems that this 
distinction got lost somewhere. 


This appears to be the main tssue 
affecting the growth of village councils as 
stronger people’s organizations, effective- 
ness of programmes, and confusions re- 
garding organizational structure and func- 
tioning at different levels. Over the years, 
the distinction between village councils as 
people’s movement and RDO as a promo- 
tion organization got blurred. As a result, 
people's representatives and forums above 
cluster level got intermixed with RDO area 
staff; animators originally developed as 
leaders from village councils became iden- 
tified, and treated as RDO staff; the streng- 
thening of people's organization, and 
appropriate structures for the same, at a 
broader level, beyond an area, could not 
take place. Many animators and other 
staff began to equate RDO central office 
with the central decision-making forum of 
the people. 


It is this blurring of distinction that 
needs to be critically looked into and 
immediately clarified. Many other actions 
can follow from the same. 


2. This has led to cooption of people's lead- 
ership developed through investment of a 
lot of energy, resources, and time into the 
formal structure of an organization per- 
forming promotional role. Such induction 
of people's leaders in institutional struc- 
ture has perhaps not only created dual 


39 


40 


aspirations among such staff but also 
helped nurture misunderstanding among 
village council members. The duality of 
aspirations had further affected the inter- 
nal dynamics of the organization. 


On the one hand, village council anima- 
tors and cluster coordinators were acting 
as spokes-persons of the people’s organi- 
zation; on the other hand, they were sub- 
jected to, and became socialized into, RDO 
organizational structure and style of func- 
tioning within the overall context. 


Many of these persons experience ten- 
sions and conflicts within themselves 
around this duality; and, act on that confu- 
sion leading to inter-mingling of roles— 
acting as people’s representatives in staff 
meetings and as RDO staff in village coun- 
cil and cluster meetings. 


Rapid programmatic and geographic 
expansion of RDO activities in its third 
phase further accentuated this situation. 
The demands for such expansion implied 
transfer of senior and experienced staff to 
start work in new areas, recruitment of 
technical and professional staff, and a 
large number of new field staff. This 
expansion appears to have not been 
planned very well and as a result necessary 
orientation and integration of new staff 
could not take place. The new phase, third 
phase of RDO activities could not be com- 
prehended by new staff within the histori- 
cal context of the earlier two phases. And 
old staff equipped to function effectively 
in those previous phases could not be reor- 
iented to take on the responsibilities of the 
new phase. 


The growth of RDO apparently less 
than fully planned, further sharpened 
internal and external contradictions and 
conflicts. 


The organizational and personnel man- 
agement style evolved during the process 
of working in a relatively smaller area 
with focus on conscientization, organiza- 
tion, and action was not reviewed and 
changed to adjust to the requirements of a 
large organization taking up development 
programmes also. Even the older staff was 
not prepared or trained in planning, 
proposal developing, monitoring etc. 


The structures of decision-making, par- 
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ticipation, mutual interaction and consul- 
tation that were appropriate during the 
previous phases were no longer fully rele- 
vant in the changed context. Similarly, 
planning, budgeting and control systems, 
previously appropriate, did not get revised 
in light of the new demands of tasks and 
people. es 


The strategy adapted to cope with the new 
programmes in expansion phase was to 
introduce the technical team model. Such 
a change in structure in a social organiza- 
tion needs to be carefully discussed so 
that the people involved could internalize 
it and it contributes to enhancing the 
effectiveness of the organization. Some- 
how it seems there is a lack of clarity about 
the objectives of such an important change 
in structure. 


As it is explained in some of the docu- 
ments, the reason for such a change was 
dual in nature, i.e., to deal with the grow- 
ing regional feeling and to cope with the 
skill: development requirement of new 
thrust. The new result was resistance 
from the people and older staff in Areas. 
This change could be fully effected in only 
one among the three old Areas, and 
another Area developed a via media of 
maintaining people’s structure alongwith 
the technical team model. 


The changes brought about in this 
structure appear to have contributed to 
the further blurring of the boundary 
between people's orgapization and promo- 
tional organization. The new demands 
perhaps necessitated changes in RDO staff 
structure, but the net effect was change in 
peoples’ organizational structure. 


The major impact of such a change was on 
horizontal communication among people's 
bodies, which seems essential for consoli- 
dation of people’s movement. With the 
introduction of the technical team model, 
no natural form is left fer the village coun- 
cil members of one village to meet their — 
counterparts from another village in the 
same cluster/Area at their own initiative. 
The only communication channel left 
open is that through RDO staff. This has 
placed the responsibility of running and 
consolidating people’s organization on 
RDO without much participation of peo- 
ple’s leaders. 


Another effect of this change was felt 
on those who were village council members 
in past and were first made animators and 
then regional coordinators, or on those 
who were village council leaders first and 
then coopted as regional coordinators. 
These people, when asked to acquire some 
or the other skill in order to change their 
role to a technical team member, found it 
difficult to cope with it in a short span of 
time available. As a result, it appears to 
have overloaded them in terms of cover- 
age; also it did not match their personal 
goals based on dual aspirations referred to 
earlier. The resistance caused by this pro- 
cess of change has affected the functioning 
of the organization. This has also led to 
mistrust and suspicion among the staff at 
different levels, and polarization of differ- 
ent categories of people in RDO and vil- 
lage councils. 


The organizational and personnel man- 
agemenit style referred to earlier, in the 
highly polarized and changed context of 
these developments, led to a decline of 
democratic norms of organizational func- 
tioning. The role of a few committees left 
with people's representatives, i.e., steer- 
ing committees, in decision-making grad- 
ually declined. The decision-making pow- 
ers, for all practical purposes, got con- 
centrated at the top. Detailed information 
regarding planned proposals and budgets 
initially used to be taken to the village 
council and discussed in order to set up 
democratic norms in the organization. 
They had stopped even reaching the staff 
now. This style of management develeped 
in all aspects, including personnel, finan- 
cial, administrative and inter-personal 
aspects. 


Most of the Area coordinators have also 
tried to imitate the same style in their 
Areas, which has made things worse. The 
only mode of information flow developed 
due to such a style if reflecting a “closed 
door policy”, wherein the policy makers 
knew the various related issues, while all 
others were in dark. As things stand now, 
the flow of information has become very 
restricted and personalized. This has also 
contributed to speculation, rumour, dis- 
tortion, and inter-personal mistrust and 
distancing. The prevailing system of in- 
formation flow and decision-making could 


not cope with changing demands of the 
new phase, and over-centralization became 
the response to the resulting chaos and 
confusion. 


The development of various economic 
and social programmes did not occur on 
the explicit premise of continuously streng- 
thening people's organization and move- 
ment. Asa result, many economic and skill 
building programmes have been initiated 


in.such a manner and at such pace that the 


promotional role of RDO (in conscientiza- 
tion and mobilization) got gradually re- 
placed by service-provider role. 


The loan programme, in particular, 
became the focal point of village councils, 
and its expansion, as opposed to that of 
village council membership and strength, 
took precedence. Evolution and use of var- 
ious SOCIO-economic programmes in sup- 
port of strengthening people's organiza- 
tion and movement somehow got diffused. 
And, as a result, programmes became the 
primary task, not the village councils and 
their higher level structures. 


The programme got conceived and 
implemented in all the Areas—old and 
new—almost uniformly, without refer- 
ence to specific requirements and needs of 
a particular village council, region or Area. 
The stereotyping of programming had 
further reduced the possibility of utilizing 
the programme to strengthen people's 
organization. 


In sum, RDO experience demonstrates 
the viability, and even desirability, of 
building people's organization, and village 
councils as their base structures. The pro- 
grammes and structures of RDO promo- 
tion organization need to be distinguished 
from the people’s organization and move- 
ment. It is in rearticulation, and conse- 
quent reformation, of this strategy that 
the future direction of people's organiza- 
tion and RDO as a promotional organiza- 
tion should be attempted: 


These observations were discussed at 
length during the meeting and there was a 
general agreement on all the points. It was 
resolved in that meeting to translate the 
whole report in Hindi and discuss it at 
various levels in the organization. Plan- 
ning of the actions to be taken was dif- 
fered in this light, to be done after the 
discussion. 
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The intervening agency did not receive 
any communication after this, hence its 
role came to an end at this stage. 


This experience brings out a number of 
issues related to the understanding of par- 
ticipatory evaluation. These issues on one 
hand highlight some essential points to be 

followed in a participatory evaluation 
exercise and on the other illuminate some 
limitations of participatory evaluation. 
We are attempting to put down some of 
these issues here for the purpose of 
further discussion. 


Issues 


Explicit statement of objectives of a pro 
posed participatory evaluation exercise by 
all the concerned parties (constituencies) 
related to the organization's work 1s 
essential for carrying out the participatory 
process. Any room for possible reinter- 
pretation of objectives in between or 
imposition of new objectives by one or a 
few parties alone is bound to hamper the 
participatory process. Hence, it is better to 
initially spend some more time in setting 
the objectives, explanation and interpre- 
tation of each objective etc. with all con- 
cerned parties, in order to ensure uninter- 
rupted participatory process later on. 


Working out a detailed “memorandum of 
understanding” type document acceptable 
to all concerned is also essential, however, 
bureaucratic it may sound on the face of 1t. 
Such an understanding may entail roles 
and responsibilities of each party and 
individuals involved in participatory eva- 
luation exercise from each party. It may 
also incorporate understanding regarding 
communication of all important decisions 
taken in the organization during the time 
of participatory evaluation exercise to all 
concerned. On the whole, based on the 
merits of the case in hand, it may incorpo- 
rate all those details regarding the under- 
standing among concerned constituencies 
which have potentials of distorting or 
interrupting the participatory process. 


This practice is quite common in sec- 
tors other than ours. Due to our utter 
dislike for structured and formal function- 
ing we tend to generally ignore working 
meticulously on this aspect. Since, most 
organizations in our sector also operate in 
a multiple constituency system and the 
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participatory evaluation process entails 
involvement of most of the constituencies 
the organization deals with, it ts always 
better to create a clarity of understanding | 
at the outset. The elements of misunder- 
standing, mistrust and confusion among 
the concerned constituencies cropping up 
in between the process defeat the whole 
purpose; hence it is essential for us to 
come to terms with this bit of formalism. 
Our experiences in recent years indicate a 
welcome trend of organizations initiating 
participatory evaluation on their own, 
without anybody asking them. This trend 
needs to be encouraged further. 


Another issue relates to strings attached 
or stakes involved in evaluation for the 
organization and its impact on participa- 
tory evaluation. The participatory evalua- 
tion is an exercise of critical reflection 
leading to strengthening the vision, organi- 
zation and programmes. The situation of 
no strings attached or no stakes involved 
will doubtlessly be the ideal situation for 
such critical reflections. But in practice 
such ideal situations often do not exist. 
Hence the participatory evaluation faces 
the challenge of coming to terms with or 
coping with the situation of a few strings 
attached or some stakes involved in any 
proposed exercise. The experience shows 
that the two points mentioned above, 
regarding explicit statement of objectives, 
and formulating memorandum of under- 
standing, do help in coping with such chal- 
lenge. It was found that such efforts may 
yield desired result at the top level of the 
organization which on its own do not per- 
colate down to various levels. Hence some 
cautious, deliberate and effective efforts 
need to be built in the process itself; so 
that the restricting effect of stakes in eva- 
luation on the process of critical reflection 
could be minimized at all levels of the 
organization and among all constituencies 
involved. 


The environment in the organization 1s 
another issue of concern in the context of 
participatory evaluation. The case pres- 
ented here had an organizational envir- 
onment of tension and mistrust which got 
polarized during the process of evaluation 
due to external factors and leadership 
response to those factors. Experience in 
this case and some other cases shows that 
in such a situation the process of participa- 


tory evaluation can be very helpful in cop- 
ing with the situation if the organization’s 
functioning was non-bureaucratic and 
Organization’s structure was non-hierar- 
chical. But in a hierarchical and bureau- 
cratic Organization even in normal cir- 
cumstances it becomes difficult to carry 
Out a participatory process. Though it is 
hard to say at this stage whether it is a 
strength or weakness of participatory 
methodology, but certainly coping with 
the inherited resistance of bureaucratic 
and hierarchial system is a challenge 
which we face in varying degrees in var- 
ious experiences. 


If we agree that for any participatory eva- 
luation exercise, it is essential to result in 
action for change, the issue of organiza- 
tion's ‘willingness to act’ becomes very 
important. In the absence of such a will, 
even a participatory exercise will tend to 
become a routine; particularly if it deals 
with organizational issue of perspective, 
structures and work culture etc. The action 
for change in this context amounts to 
organizational development intervention 
and thus, even the role of intervening 
agency becomes crucial. So the willingness 
and readiness of the organization and 
acceptance and competence of interven- 
ing agency holds the key for such desired 
organizational development intervention. 


In a participatory evaluation exercise, cho- 
ice of appropriate techniques of data/in- 


formation collection\is another crucial area. 


Perhaps the rationale for choice of tech- 
niques should be promoting involvement 
of the people at various levels of the 
organization's work. This makes it imper- 
ative that choice for differential levels has 
to be different. The selected technique has 
to be conducive for the level it is going to 
be used at. Hence, flexibility in choice of 
technique is detrimental for a successful 
participatory process. 


This requirement places extra demands 
on the facilitating team of having com- 
mand over all the techniques, so that it 
could use appropriate ones, whenever and 
wherever required. 


ANIL K CHAUDHARY 


OFADEC, Senegal 


Introduction 


In December 1980, I was recruited by the 

Office Africain pour le Developpement et la 
Cooperation (OFADEC) for coordinating the 
activities of their training department at the 
level of village cooperatives which the said 
NGO was carrying out in the eastern part of 
Senegal. At the same time, six other fresh 
university graduates like me.were recruited to 
serve in that department. 


The OFADEC’s objective was to strengthen 
the department, the importance of which 
seemed to be vital for the future of the pro- 
gramme, after a three-year experience. After 
all, OFADEC, which had launched its activities 
in the eastern part of Senegal in 1977 intended 
to promote the participation of the rural popu- 
lations, whom it was assisting, in the process 
of their self-development within the frame- 
work of an economic and social approach. 


The context of the OFADEC’s intervention tn 
Senegal was characterized by the cycle of 
drought that had become common in the 
Sahel, and which made the conditions of living 
of the rural populations, who constitute the 
majority, very hard. For the OFADEC, the cho- 
ice of the eastern part of Senegal was due to the 
fact that the region is a landlocked one, situ- 
ated at 500 kms from the capital city. The 
area's economy is dominated by a winter agri- 
culture and livestock. The region in question, 
which benefits from River Gambia, borders 
Mali, Gambia, Guinea and Guinea Bissau. 


Traditionally, there were also seasonal popu- 
lation movements. The scarcity of rains had 
accentuated the rural to urban areas exodus, 
especially towards Tambacounda, the regional 
headquarters (450 kms from Dakar). 


The objective of OFADEC was to stop that 
rural exodus by establishing a modernized 
agriculture with a system of simple irrigation 
and physical planning. 


The strategy applied by OFADEC with regard 
to the promotion of an integrated community 
development was based on the creation of vil- 


lage cooperatives through a three phase 
process: 


1. The first phase goes from the creation of 
the cooperative and the development of 
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the agricultural parameter up to the first 
harvest. At this stage, which usually lasts 
for 18 months, all the costs are covered 
by the NGO which in addition provides 
some foodstuff and a small amount of 
money (about 10 dollars) per month. 


2. The second phase starts with the first 


harvest and it is characterized by the 
establishment of a co-management sys- 
tem between OFADEC and the coopera- 
tive, whereby the costs are shared. At 
this stage, the cooperative contribution 
increases proportionally to the growth of 
its activities while the NGO's contribu- 
tion decreases. 


3. The last phase is the phase at which the 


NGO withdraws its support to the coop- 
erative that has reached the stage of 
maturity. It is then assumed that the 
cooperative has sufficient means and 
skills, to support. itself, and all expenses 
incurred by OFADEC towards it are 
refundable. 

The whole process should last a maximum 
of five years. Within the irrigated parameters 
the main crops are: banana (the main cash 
crop), irrigated maize plantations and vegeta- 
bles (off season). 


OFADEC had established many departments 
in order to carry out its activities: 


—the production department (The produc- 
tion department was in charge of techni- 
cal aspects with regard to crops — physi- 
cal planning, installation of tractors and 
motor pumps, selection of crop varie- 
ties...), 

—the administration and finance depart- 
ment, 

—the health department, 

—the training department. 

The training department had four func- 
tions: | 

—to ensure cooperative training for all. 
related organs, so as to make them more 
democratic and participatory, 

—to facilitate literacy among cooperators, 

—to facilitate the technical training of 
cooperators (management, tractor driv- 
ing, handling of motor pumps, utiliza- 
tion of fertilizers ...) in collaboration 
with other OFADEC departments. 


Thanks to the assistance of the Interna- 
tional Labour Office (OIT), the said training 
department was established in 1978, with the 
aim of facilitating the peopte’s participation in 
the OFADEC activities. ' 


Nevertheless, that initiative did not last for 
long, because the approach was quite classic 
and based on programmes which had vety 
little in common with the local conditions. 


Thus, by recruiting young people, inexpe- 
rienced but motivated and whose political 
ideas were considered radical, OFADEC 
intended to do away with the type of classic 
training so as to adopt a new and more partici- 
patory approach. 


In 1980, when We joined OFADEC, its activi- 
tie$ covered six villages — the same number of 
cooperatives — even though each cooperative 
had only about one hundred members; but the 
total number of people who were directly or 
indirectly concerned amounted to more than 
5000 persons. 


The six trainers’ team, acting at the six vil- 
lages’ level, started their work by an exercise of 
evaluation of the cooperators’ needs in the 
field of training, on the one hand, and with a 
study of the local sociological conditions. 


One of the objectives of that evaluation was 
to identify the themes for the cooperators’ 
training. It had also to establish up to what 
level the management system established by 
OFADEC had been accepted by the populations, 
or in other words whether the community sys- 
tem established within the cooperatives was 
really functional. 


The procedure used within the framework 
of that evaluation was after all classic: the 
trainers’ team was, in the course of the exer- 
cise, to familiarize itself with the villages 
where its members were acting and use a ques- 
tionnaire during interviews with cooperators. 
It should be indicated that all cooperators were 
interviewed in each village. 


The whole process of this evaluation exer- 
cise lasted for three months and hereafter are 


its main results: 


a. Almost all the cooperators considered 
the OFADEC management system as a 
harmonious system that took into 
account their traditions of community 


life. 


b.  Allagreed with three phase progressive 
scheme put forward by OFADEC and were 
ready to be trained in various fields so as 


to take vver the OFADEC functions. 


c. Functional literacy, in French, was one of 
their main concerns. 


Those idealistic conclusions of the evalua- 
tion did satisfy the OFADEC management and 
the trainers’ team. But it very soon became 
clear that they did not correspond to the 
reality. 


Indeed, after about six months of cohabita- 
tion with the cooperators, a number of factors 
indicated the complexity of social relations at 
the level of supported villages and co- 
operatives. 


It thus became clear that cooperatives were 
rather the affair of the dominant class of the 
village where they were established and that 
the influential peopie of the said village had 
more say within them than their “foreign” 
members. 


Women and young people were also sys- 
tematically excluded from their decision mak- 
ing process. 


Well, there was some representation, but 
only nominal, of foreigners and women at the 
various cooperative boards of directors, and at 
such instances as general assemblies, it was a 
matter of approving decisions taken in other 
quarters. 


On the other hand, it was clear that the 
system of a collective perimeter established by 
OFADEC did not strengthen the sense of com- 
munity, because of its poor technical perfor- 
mance. On the contrary, it rather tended to 
aggravate inegalities by enabling big families, 
especially those of influential people to accu- 
mulate more working days, which constituted 
the basis for the sharing of production. Those 
who could also afford to employ casual workers 
to work on their behalf continued to benefit 
from their cooperative membership: sharing 
of profits, distribution of foodstuff... 


Generally speaking, the same types of pro- 
duction relationship that generated social dis- 
parities at the village level were being repro- 
duced within the collective perimeter. It was 
thus not’surprising that while the acreage pro- 
duction was decreasing at the perimeter level, 
the individual (or family) farms were more 
looked after by the rich peasants who were the 
land owners—the landless having of course no 
right at those. 


Given such circumstances, there seemed to 


be a double language, one for foreign people, 
and giving the impression of harmony and 
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responding to the interests of influential peo- 
ple, and the other, which reflected the objec- 
tive conditions, but that could be perceived 
only after a period of cohabitation with the 
peasants. It seemed to satisfy everyone s inter- 
ests: while visiting cooperatives OFADEC and 
funding agencies felt satisfied by the tune of 
community speeches, and the landless did not 
want to jeopardize their precarious situation 
by denouncing the influential people. 


Once the trainers’ team became aware of 
that situation, a new strategy was adopted in 
order to make the evaluation more objective. 


The said strategy was based on the following 
assumptions: 


1. It was first necessary to gain the pea- 
sants’ confidence so that they may 
express freely their opinion and their 
analysis. 


This was essential and it was not 
automatic; because, as indicated above, 
the trainers were young educated people 
from urban areas, which in itself did not 
attract the peasants’ trust. Apart from 
that, like most of the OFADEC personnel, 
their socialist political orientation made 
them naturally believe that the system of 
collective production in place was more 
appropriate than the private property. 
The question is: What does one do to 
ensure the real participation of the 
peasants? 3 


2. Because of its philosophy, OFADEC had 
to aim its activities mainly towards the 
disadvantaged groups, particularly the 
landless, women and young people. 


Indeed, even if the influential people 
had played some role with regard to the 
introduction of OFADEC at their village 
level, that did not mean that they should 
control the cooperatives. 


What should be done to ensure the 
implementation of democratic principles 
within cooperatives? 


3. Peasants perceived OFADEC as a charity 
organization, that came with its own 
development scheme and which they 
should avoid to displease, so as to con- 
tinue getting support from it while at 
one and the same time developing 
strategies that did not much correspond 
to stated principles, but which responded 
to the cooperators’ fundamental needs. 


What was to be done so as to get OFADEC 
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put into practice its own participatory princi- 
ples by enabling the cooperators decide them- 
selves the best way to ensure their develop- 
ment? 


Asa response to those various questions, the 
following proposals were put forward by the 
training team: 


1. To break the barrier between them and 
the cooperators by participating more 
actively in their daily activities, through 
promotion actions at the village level, 
especially among the youth (cine-clubs, 
football matches...). 


2.  Toidentify the most active people among 
the disadvantaged and to establish strong 
human links with them. 


3. The recruiting of awoman by OFADEC as 
a liaison officer with cooperatives. 


To do away with questionnaires and to 
encourage discussions in general assem- 
blies where issues of general interest are 
debated and following which resolutions 
adopted are impiemented. 
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5. To encourage exchanges between coop- 
eratives so as to promote a sound emula- 
tion and the circulation of informatron. 


6. To encourage young people from urban 
areas to get affiliated to cooperatives, so 
as to minimize the local influential peo- 
ple’s influence. 


7.  Toorganize an exchange programme for 
and between cooperators and peasants 
from various regions. 


At the end of one year during which these 
proposals were implemented, a lot of positive 
results were registered witH regard to the par- 
ticipation of peasants in the sound manage- 
ment of their cooperative. Of course, with 
regard to villages, the progress was more or 
less apparent, but generally speaking large 
meetings (general assemblies) played more 
and more the role of forums where collective 
problems were solved and the position of : 
influential people was jeopardized following 
elections that brought to the fore the most 
talented people of the cooperative movement. 


The reorganization of the production sys- 
tem brought about the most outstanding result, 
with the acquisition of individual farms. 


Furthermore, following the improvement 
of their income due to the new production 
system, the cooperators could contribute more 
to common interest projects such as: health, 


education, recreational projects, etc. 


All that was achieved through the participa- 
tion of all interested parties. Cooperators were 
more and more resolving their own problems 
and they did hesitate to propose to the OFADEC 
cadres solutions to technical problems that 
looked too complicated to solve. 


However, faced with a situation whereby 
they were more and more losing their advan- 
tages, the local influential people reacted by 
activating their political and administrative 
alliances against OFADEC, especially against 
the trainers, brandishing them as “politicians”, 
and as ‘subversive elements”. The landless 
people were also accused of exploiting the best 
areas, often simply because they were making 
the best use of it. 


At this stage OFADEC found itself in a 
dilemma: it had either: 


—To strengthen the innovative trend ini- 
tiated in collaboration with the poor pea- 
sants, and if necessary create new vil- 
lages, in case the ihfluential people of the 
old ones did not abandon their hostile 
attitude towards the landless cooperators 
and the trainers; or 


—To readopt the previous approach, so as 
not to displease the influential people 
and the politico-administrative apparatus. 


After rough discussions both at the coopera- 
tive level and within OFADEC, it is the second 
alternative that was finally adopted. The train- 
ing department was consequently abolished in 
1983 and the training team dismissed, apart 
from one trainer who was serving a coopera- 
tive composed of members who had all come 
from the urban area. 


Conclusion 


In 1986, I had the opportunity of visiting the 
OFADEC villages. It was within the framework 
of an evaluation study funded by the Ford 


Foundation. 


A new training department with a more 
classic approach had been reconstituted. The 
advantages inherent to the small individual 
farms ownership persisted. Nevertheless, that 
affected more and more the community char- 
acter of the programme. 


Generally speaking, OFADEC was going 


through a difficult period, because of funding 
problems. 


But, finally, what was reassuring was that 


the peasants had become more demanding 
with regard to their rights and were deter- 
mined to participate, individually and collec- 
tively, in the development process with or 
without OFADEC. 


That experience was very useful to me and 
to my colleagues of the training team. First, it 
was the first long contact with the rural popu- 
lations, and moreover, apart from certain 
general ideological principles the theory of 
participatory research was unknown to the 
development agents, as it is still the case in 
most Francophone countries of West Africa. 


Hereafter, some of the many lessons I got 
from that experience: 
1. The simple protession of the partucipa- 
tory concept, by cadres as well as by 
development agencies is not enough. 


It is a process throughout which all the 
parties concerned must constantly review 
their viewpoints. It is a mutual learning 
process where in most cases the cadre 
and/or the development agency have 
more to learn. 

2. Lhe populations, especially the rural 
populations, do develop strategies in 
order to find solutions to their problems, 
instead of being passive with regard to 
events, and they always do it in a partici- 
patory manner. 

‘The cadre or cne aevelopment agency 
must take that into account, so as to con- 
sider, in collaboration with them, wit is 
possible or not in view of the experience 
acquired by peasants in a particular field 

Any positive changes or any other 
innovation cannot succeed without due 
collaborations with the populations. 


3. Areal participationof the populations in 
a development action introduced from 
without supposes a certain number of 
conditions, such as: 

a. A deep knowledge ot the objective 
social conditions, i.e. the existing 
social strata and consequently of the 
economic, social and cultural pres- 
sures. | 


b. Thedeepest possible integration with 
the populations, particularly with 
the disadvantaged. 

c. A valorization of all the means that 
facilitate the expression of different 
opinions and viewpoints, with regard 
to socio-cultural realities: general 
assemblies, group discussions, popu: 
lar plays... SEY DOU SALI 
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FARMDEYV, Inc., Philippines 


The FARMDEV 


The Forum for Approaches in Research, Media 
and Development, Inc. (FARMDEV) Is an edu- 
cation agency serving the basic communities in 
the four provinces of Agusan and Surigao, 
Mindanao, Philippines. Through its Micro- 
Media Programme, Popular Education Pro- 
gramme, Resource Development Programme 
and Partnership and Linkages Programme, 
FARMDEV reach out to the marginalized sector 
in Northeast Mindanao (NEM) and participate 
in their struggle for self-determination. 


Since 1981 it had helped groups of farmers, 
labourers, women, tribal people and other sec- 
tors in their trainings and education towards 
attaining critical consciousness. FARMDEV 
aims to contribute to the education of the peo- 
ple so that’ they can analyse and understand 
their own situation and most of all the forces 
that shape these situations. 


FARMDEV values alternative and liberating 
education—an education towards a human and 
democratic Philippines society via alternative 
media, research, education and development 
projects. 


The context of evaluation assistance 


As early as 1982, FARMDEV started its evalua- 
tion assistance to people's programmes and 
organizations in Northeast Mindanao — 
health programme, justice and peace pro- 
gramme, young people's activities and farmers 
organization. The succeeding points are the 
documentation of the process and learnings of 
FARMDEV in its evaluation assistance to peo- 
ple’s programme and organizations as well as 
its experience in evaluating its own work. 
These experiences are limited to a four to five 
days’ evaluation sessions of the requesting 
programme and organizations. 


In the case of FARMDEV’s work in NEM, the 
following realities of people s programme and 
organizations are worth mentioning: 

1. People’s development programme and 
organizations in NEM came into being to 
respond to the needs of the basic sector 
who are victims of human rights viola- 
tions and who aspire and struggle for a 
more human and democratic society. 


People’s programmes and organiza- 
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tions in NEM are operating in the context 
of the socio-economic and political reali- 
ties of NEM in particular, and of the 
whole Philippines in general. The stand- 
point is clear, that is to work and be one 
with the basic masses who struggle for 
selfdetermination. 


People’s development programme and 
organizations are manned by persons 
who have strong commitment for change 
but have lesser skills to do the work sys- 
tematically. Specific manifestations of 
this reality are the following: 


2.1 Lack of definite long range plans: 
The ultimate goal of people's pro- | 
grammes and organizations Is clear. 
However, there are difficulties in 
planning concrete and interrelated 
steps to achieve this goal. Most often 
activities conducted are based on 
requests and relevant events. These 
seasonal’ activities unfold the prag- 
matic nature of the programme of 
organization. Consequently, there 
are times that the workload 1s too 
great that workers will do overtime 
work. And when the ‘season’ 
changes, workers will become idle 
and are confused what to do next. 
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Indifferent attitude towards profes- 
sionalism in running programmes, 
methods of basic systematization and 
work simplification schemes are 
often times seen as encroachment of 
bourgeois tendencies. Professional- 
ism in operations is not so enthusi- 
astically adopted. 
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Documentation and report writing, 
a burden: Stereotype notion states 
that documentation and other paper- 
works are only for the professionals 
and not for those who are immersed 
in working with the people. How- 
ever, further investigation reveals 
that the real reasons to this attitude 
are the following: lack of thorough 
understanding on the importance of 
documentation in working with the 
people, lack of appropriate skills to 
do it, and the security risks given the 
political conditions in the area where 
those who are working for people's 


development are labelled as sub- 
versives. 


It is very important to note how- 
ever that people’s programmes and 
organizations are very rich in direct 
experiences. 


2.4 Multi roles of a single organization 
vis-a-vis its nature and direction: It 
is a common observation that peo- 
ple’s-organizations and programmes 
sometimes launch activities contrary 
to their nature. In this case duplica- 
tion and unnecessary overlapping of 
work happened instead of synchron- 
izing efforts and maximizing re- 
sources. This is not surprising 
because these programmes/organt- 
zations are very much affected by the 
enormous needs and demands of 
their target beneficiaries as well as 
with the broad and general descrip- 
tion of their own work. 


2.5 Hazy coordination work with other 
related organizations or programmes: 
This usually occurs in organization- 
institution relationship. Per expe- 
rience, an institution imposes activi- 
ties for implementation to the orga- 
nization without actually discussing 
with them the comprehensive plan. 
The organization on the other hand 
reacts to this approach, since they also 
have their own needs not considered 
by the service institution. Because of 
this problem of undefined coordina- 
tion, sometimes the two will launch 
the same activity and it is the benefi- 
ciaries who will be confused in the 
process. 


2.6 Readymade plans for implementa tion: 
This contradicts the liberating aspect 
of the humanization process in devel- 
opment work. In most cases, imple- 
mentors do not participate in the con- 
ceptualization process sO that they 
could implement plans according to 
what was explained to them and 
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these plans. This result is a strained 
relationship between the conceptual- 
izers and the implementors since the 
former will demand a to the letter 


implementation and the latter for 
close guidance. If the implementation 
fails, the bulk of the blame is given to 
the implementors. 


We 


Evaluation, a built-in mechanism of peo- 
ple’s programmes/organizations: In the 
daily existence of people's organizations, 
evaluation plays an important role. Con- 
sciously, evaluations are made part of the 
work and are exercised: 


a. every after a major activity 

b. monthly 

c. quarterly 

d. year-end, in the form of summing-up 
of the whole year operations. 


Integral in this process is the self-evaluation 
wherein workers reflect on their experiences 
and processes and lay down recommendations 
for rectification. 

The above realities of people’s development 
programmes/organizations as well as the socio- 
economic and political conditions provide the 
context of FARMDEV’s evaluation assistance to 
these groups. 


Objectives of evaluations 


People’s programme and organizations 
requesting for evaluations have the following 
objectives: 


1. Tosum up past experiences to be able to 
draw out lessons and recommendations 
for future actions. 


2. To define the nature of the organizati- 
on/ programme and streamline activities. 


3. Toset up direction of the programme/ 
organization and lay down long range 
objectives. 


4. To resolve management programmes 
and prepare for a basic management 
training. 


5. To level-off understanding of the whole 
programme/organization among staff 
members, or workers towards a unified 
view in running the whole programme/ 
organization. : 


As for FARMDEV as the facilitator, the main 
objective of the whole process is education 
towards people's empowerment. Evaluation ts 
seen as an opportunity for the people to learn 
to express themselves, to objectify their own 
experiences and to analyse their situation and 
the forces shaping these situations. 
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Conducting evaluation sessions 


FARMDEV conducts evaluation upon request 
by people's programme/organizations. The 
FARMDEV evaluators only facilitate the pro- 
cess so that the participants can objectify their 
work experience. 


The evaluation follows the following phases: 


I. Pre-evaluation session 


1. Discussion-sharing with the head of the 
organization/ programme. 

2. Discussion-sharing with the _ staff 
members and other workers related to 
the programme/organization to be 
evaluated. 

3. Informal sharing with the other organi- 
zations or programmes who have coor- 
dination work with the organization. 

4. Informal feedback gathering from a few 
of the target beneficiaries. 

5. Study of all available documents of the 
organization / programme (particularly 
minutes of meetings and assessment 
results). 

6. Brainstorming on the objectives and 
direction of the particular evaluation 
together with the key persons of the con- 
cerned organization/ programme. 


The pre-evaluation session is the prepara- 
tion phase. Here the facilitator familiarizes 
itself with the organization’s workings. This 
phase helps her/him facilitate a down-to- 
earth discussion with the group, provides her/ 
him with the preliminary data and assumes 
her /him that the group will be talking in the 
same wavelength during the actual evaluation. 


If. Evaluation proper 


1. — Levelling-off of expectations and objec- 
tives of the session by all those who will 
participate in the evaluation. 


2. Evaluation proper 
2.1 review of objectives and plans, 


2.2 presentation of data in the forms of 
activities conducted, persons in- 
volved, time-frame, concrete results, 
problems encountered, solutions to 
problems, etc., 


2.4 discussion, clarification and valida- 
tion of data presented, exchange of 
views and opinions, 


2.4 synthesis of the discussion, drawing 
out of learnings and discoveries 
(group work), 
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2.5 projections’ setting. 


The evaluation proper exhausts the partici- 
pation of everybody who is involved in the 
operations of the organization. To achieve 
this, the facilitator injects reflective questions 
and often times serves as the devil’s advocate 
in order for the participants to objectify their 
experiences and learn lessons from it. 


III. Post-evaluation (follow-up stage and 
the planning process) 


After the evaluation session, the next stage 
will be consultancy stage. The organization/ 
programme will seek the aid of the facilitator 
in operationalizing certain recommendations 
made during the evaluation session. The cul- 
mination of this activity (after a two-day 
break) is the planning process which is being 
done by everybody with the guidance of the 
facilitator who prepares the foundation by dis- 
cussing with the participants the mechanics of 
planning. 


Steps: 


1. Study of the present situation (of the 
“environment, socio-economic and politi- 

cal situation of the needs of the target 
beneficiaries, of the capacities and limi- 
tations of the programme/organization), 


2. Clarification of the nature of the people’s 


programme/organization and direction 
setting, 


3. Planning (setting-up objectives, plotting 


activities, formulating indicators of suc- 
cess, etc.). 


The post-evaluation phase is the outcome of 
the evaluation session proper. After the peo- 
ple’s programme/organization is being con- 
fronted with the work experience—strengths, 
weaknesses, lessons—a desire to straighten 
things up to become more effective to the 
beneficiaries is brought about. 


Observations/discoveries/learnings 


It is important to note that all these evaluation 
experiences ended up in basic management 
trainings. This is so because of the desire of the 
participants to be equipped with the skills so 
that they can improve their self-evaluation 
activities and do their own programme evalua- 
tion in the future more effectively. 


During the evaluation proper, workers in 
people's programme/ organizations realize that 
in the course of their working with the people, 
they have no strategic plans, that is to look 
ahead and view things beyond what happens 
today; they tactically analysed activities and 
events and employ stopgap solutions in solv- 
ing problems instead of hitting the key prob- 
lem which is crucial in the major shifts of the 
organization/ programme operations. 


In all these experiences, it has been the guid- 
ing principles of FARMDEV to involve as 
much as possible all those concerned including 
the beneficiaries. Participation is the key work 


in evaluation work as well as in the other work 
of FARMDEV. 


Above all, FARMDEV is concerned with 
the humanization process, both of the benefi- 
ciaries and that of the workers. In every evalua- 
tion conducted, the following questions are 
always incorporated as a check of the process: 


—'‘‘Are the persons doing development 
work developed in the process?” 


—“‘Are the methods employed in the work 
for development liberating both the 
beneficiaries and development workers?” 


—"Are the target beneficiaries given the 
chance to express their views and analy- 
sis on certain things/events affecting 
their own lives?” 


Finally the ultimate questions asked during 
the evaluation-of people’s programme/organ- 
ization, to measure results are: 


—"What happened to the target benefici- 
aries with the activities being launched?” 


—"If you will phase out the programme, 
can the beneficiaries continue the work 
you have started?” 


—‘‘Are you confident that the beneficiaries 
can already withstand any difficulty in 
the future without the programme/organi- 
zation?” 


These questions usually will silence the par- 
ticipants and will bring them to reflection and 
self-examination. From here, a new lit of 
enthusiasm to be with the basic masses is 
renewed, new strength and courage are built 
up, and a new perspective is developed. Devel- 
opment work is now seen as 4 task for the 
beneficiaries and for the workers in solidarity 
with all the others who struggle to attain a 
more humane social order. 


In our evaluation work as well as in all our 
trainings, we make it a point to comprehen- 
sively weave liberating education principles in 
the whole process together with the partici- 
pants. We create conditions to — objectify 
experiences for the programme/organization © 
to critically analyse their own work and do 
something with the discoveries, learnings, and 
recommendations that will surface during the 
whole process. 


As facilitators to these work experiences, © 
FARMDEV together with evaluation partici- 
pants crystallized the following learnings: 


1. Internalization of the nature and direc- 
tion of the programme/organization and 
setting up clear goals based on sound 
analysis of the environment and needs of 
the beneficiaries wili facilitate a focused 
discussion and indepth learning processes 
during evaluations. 


2. Providing opportunities and conditions 
for development workers to create and to 
liberate themselves in the process of 
working for the development of people 
is part of the whole humanization pro- 
cess. This could be concretely done 
through a participatory process in the 
conceptualization and planning, imple- 
mentation and evaluation. 


3. Evaluation is a learning process, a power- 
ful tool for empowerment and a venue 
for man to reflect as man, as subjects not 
as objects to be manipulated, as persons 
who exercise self-determination and 
master conditions. 


4. The success of evaluation as a learning 
process depends upon the openness of 
the persons participating in the evalua- 
tion and the facilitators’ internalization 
of the programme, of the socio-economic 
and political situation where this pro- 
gramme/organization is contextualized 
and her/his genuine participation and 
commitment to assume responsibility 
not only in the process but also in the 
results. 


MELODIA TABORNAL-ARBAN 


A MANGYAN CASE, Philippines 


Indigenous people in the Philippines number 
at least 4,000,000, equivalent to 12 per cent of 
the country’s total population. In one of the 
country’s big archipelagos are to be found a 
group of ethnic tribes collectively called the 
Mangyans. Their simple and peaceful living is 
being threatened by exploitation of opportu- 
nist groups and individuals. They are now 
beginning to recognize the problems besetting 
them which are actually no different from the 
problems faced by their fellow indigenous 
brothers and sisters throughout the country. 
Initial and decisive local actions in response to 
these problems have been taken through their 
community leaders. They were able to articu- 
late their basic demands along their four 
points of struggle: 


1.  Self-identity 
2. Ancestral lands 


3. Respect for their indigenous laws and 
culture, and 


4. Self-determination. 


In 1980, an NGO in response to their prob- 
lems and support to their struggle embarked 
on community organizing activities among the 
Mangyans. The approach employed was rural 
conscientization. Field workers were made to 
integrate/immerse in their community and 
served as facilitator for local activities related 
to organizing. To ensure the continuity of the 
organizing programme, potential leaders were 
eyed and developed. The education component 
of such an activity was implemented to the 
initiative of progressive leaders among them. 


It was then more of a sweeping or loose 
organizational structure which was actually a 
preparatory stage for the formation of a for- 
mal community organization. Assessment acti- 
vities were facilitated by the field workers who 
were immersed into their living condition. 
The purposes of such assessment were to iden- 
tify the level of awareness of the potential local 
leaders and which would then gauge the field 
workers in upgrading their level into a higher 
level of consciousness in their struggle. 


The main principle involved the drawing 
out system wherein they were made to feel free 
and feel their right to articulate their ideas and 
opinions. This process was seen as a tool for 
empowerment of these indigenous people as 
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they developed the confidence in themselves 
through active participation in assessment 
activities. 

In 1982, the NGO field workers were pulled 
out from the community while the local link 
with the NGO was maintained through the 
community leaders and other local contacts. 
There had been ongoing local initiatives in 
response to micro issues in the form of spo- 
radic community actions against illegal log- 
ging and landgrabbing. Consultations between 
the local leaders and NGO representatives/ 
organizers included assessment activities 
wherein the local leaders bring in their diag- 
nosis of area conditions and recommendations 
as well as seeking assistance from the NGO. It 
was also during this period when the potential! 
leaders recommended by the community elders 
were undergoing skills training outside their 
area under the sponsorship of the NGO. 


This has been the condition until 1986 when 
another NGO pursued the community organiz- 
ing programme and fielded their community 
organizers. The field worker adopted the 
approach employed by the former NGO. 
Assessment activities were done with the core 
group members identified among the existing 
progressive local people. The field worker acts 
only as facilitator for the core group members 
to articulate and systematize their discussion. 


The nature of organizing started with the 
follow-up of existing progressive leaders who 
were the product of the past organizing efforts 
of the former NGO in 1980 and 1981. Presently, 
community organizations have been evolved 
among the different settlements in the area, 
expanding from one tribe into two tribes. 
Plans are to include other tribes so as to 
achieve a more unified Mangyan struggle for 
their common ends (as stated earlier). 


The focus of evaluation involved in the 
analysis of organization wherein the strengths 
and weaknesses are identified and analysed | 
objectively with the leaders who then come up 
with their own recommendations for planning 
and future actions. The assessment of activities 
and organization is always linked and con- 
textualized within their basic demands. 


The impact of assessment activities included | 
the expansion in membership to the orga- 
nizations. A factor)seen was the initiatives of 


the members of the organizations who have 
undergone organizational trainings and parti- 
cipated in community (struggle) activities to 
influence other non-members to join in 
through their conversation and conferring 
about the issues they are facing. Assessment 
also brings out ideas based on their objective 
analysis of the prevalent conditions regarding 
what relevant activities are seen by them. Here 
they cite or identify possible projects that will 
serve to unite or increase the solidarity among 
-the members of the organizations. 


A major characteristic feature of the 
methods and process of Participatory Evalua- 
tion is the adoption of a structure of activities 
as a guide for evaluation. The components of 
the structure of activities are embodied along 
the five different aspects namely: education, 
social issues, organization, economic and 
health. It involves assessing of different/all 
the activities done under each aspect. Here, the 
local organizations come up with the strength, 
weaknesses and recommendations from which 
they draw out their plans for the next activities. 
The impact of activities are also assessed as to 
their bearing on their basic demands. 


The framework of structure of activities 
only served as guide; the description of their 
experience/expression of ideas is coming from 
the indigenous people themselves. 


The outcomes of evaluation activities feature 
facilitation of the evaluation of a systematized 
implementation of their activities through the 
formation of relevant strategies and techniques 
as a. product of ideas and experiences they 
articulate. Another result is the acquisition of 
additional skills on managing the organization 
on the part of the indigenous group involved. 
Still another thing noteworthy is the streng- 
thening of their commitment and a further 
deepening of the concept and essence of their 
struggle. 

The follow-up actions consist of moves based 
from the past/previous assessment and plans. 
These follow-up actions in the form of follow- 
up assessment to check on the developments 
within a certain period after the previous 
assessment has helped much in systematizing 
the activities being undertaken better, Para- 
meters are also set to gauge the capacity of an 
organization towards self-reliance. 

A recurrent issue in evaluation with the 
indigenous people are the difficulties en- 
countered in dealing with them who have 
different levels or the existence of imbalances 


in their awareness towards their common 
struggle. This is due to the presence of rela- 
tively advance and relatively (left) lagged 
behind individuals or groups in their percep- 
tion of their existing problems. 
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A STUDY OF THREE 


HELVETAS PARTNERS, Philippines 


The evaluator 


Partners in Self-Reliant Technology for 
Development (PARTNERS) is a non-stock, 
non-profit service institution based in Naga 
City (Bicol region), Philippines. Primarily, it 
was established to assist in the empowerment 
of the peasantry in the region through the 
conduct of socio-economic projects, researches 
and trainings. 


PARTNERS started in June 1986, from the 
initiative of a group of professionals who have 
had experiences in institutional (NGO) work. 
Coming from various fields of endeavour, the 
group started off with a six-month research 
project. The project used the principles of par- 
ticipatory research but was not very much suc- 
cessful due to uncontrollable external factors. 
Most of the research areas were heavily mil- 
itarized and impeded the staff from long 
period of integration with the target sources of 
information. 


In mid-1987, PARTNERS launched a partici- 
patory training project. The project went 
through the stage of identifying the training 
needs of the target beneficiaries prior to the 
conduct of the training proper. This stage 
which lasted for six months entailed series of 
structured and non-structured interviews with 
the target participants. Validated results of the 
interviews were made the bases for drafting 
the training curriculum. 


The project is ‘at its last stage. General eva- 
luation of the training and follow-ups is being 
conducted for the next trainings. 


Presently, PARTNERS is conducting a parti- 
cipatory evaluation on institutional planning 
and development. This is the case study dis- 
cussed in this paper. 


A. Background 


A Study of Three Helvetas Partners Regarding 
Institutional Planning and Development ts 
still an on-going project conducted to three 
non-government organizations in Mindanao. 
The study is basically research which is also 
evaluation in nature. 


The study evolved from the recommenda- 
tion of all Helvetas Partners in the Philippines 
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during its annual meeting in March 1987. Hel- 
vetas Swiss Cooperation for Development is a | 
funding agency which supports a number of 
NGOs in the Philippines. Helvetas has a Phi- 
lippine office. 


The general objective of the study is to draw ° 
lessons on the actual practice of institutional — 
planning and development from among Hel- 
vetas Partners. The lessons that will be derived 
from the study will serve as reference for all 
other NGOs in the country. 


For the subjects of this study, the aim is to 


‘help consolidate themselves institutionally and 


organizationally by identifying problem areas 
in their courses of operation, The study is eva- 
luative in nature because the different pro- 
cesses they -have undergone all throughout 
tHeir existence will be assessed based on cer- 
tain criteria. 


The subjects, however, will not be assessed 
and evaluated based on parameters and criteria 
set by Helvetas but on their perceived and felt 
needs. In this way, the study will benefit prim- 
arily the subjects with the hope that through 
the process, they will be able to improve their 
performance and effectivity not only in the 
programmes being supported by Helvetas but 
more in achieving their goals they have set for 
themselves. 


B. Main principles operationalized in 
the process 


The. main principle applied in the process is 
maximum participation of the subjects all 
throughout the different stages of the study. 
The respondents were not treated as mere 
source of information for which the study 
could jump from. Rather, they were treated as 
keys to the solutions of the problems confront- 
ing them in their operations. Hence, their 
involvement from the start to the end of the 
study. 


In this case, the terms of reference were 
worked out jointly by each subject and the Case 
Study Team. The subject decided on what _ 
aspects should be highlighted and be appor- 
tioned deeper analysis. They were also more 
decisive on the extent of information that will 
be divulged to the reading public. 


During integration with them, they pro- 
vided counterpart in terms of material 
resources. 


The subjects also participated in the valida- 
tion and analysis of the data. 


C. Focus of the evaluation 


‘The study is focused on institutional planning 
and development. How the subjects exercise 
the principles of institutional planning and 
development vis-a-vis their particularities and 
commonalities is the main question being 

answered. The whole of the institution—its 
orientation, programmes and organization— 
‘was studied. Emphasis, however, is given to 
the specific process undergone in their opera- 
tionalization and the contributing factors 
alongside. 


In measuring the degree of success and fail- 
ure, the targets set by the subject serve as bases 
for its appraisal—the plan-vs-actual scheme. 
Emphasis is also afforded to the impact of 
subjects’ services to the beneficiaries. 


In the final analysis, services of the subjects 
will be weighed in terms of appropriateness, 
applicability, technical and social feasibility. 
Alternatives to. the problems encountered by 
the subjects will be laid down considering all 
possible resources and constraints. 


From the experiences of these subjects, the 
role of Non-Government Organizations and 
that of the People-Based Organizations will be 
put into context. NGOs which assume services 
to the needs and problems of people’s organi 
‘zations is fast being eroded by the growing 
tendencies of people’s organizations to sum- 
marily assume the functions of NGOs. 


Incidentally, two of the subjects are people- 
based organizations which at the same time 
function like an NGO. They have their service 
arm to answer the needs and problems of their 
members. The question at stake is the perspec- 
tive of NGO work. 


D. Methodology 


The study underwent three general phases: 
Preparatory Phase in Phase I, Operationaliza- 
tion Phase in Phase II and Output Dissermina- 
tion in Phase IIL. 


In Phase I, the Case Study team was formed 
through PARTNERS. With due consultation 
with Helvetas, the study team set plans and 
strategies which was later consulted to the 


subjects upon their confirmation of intent to 
participate in the study. 


Prior to the actual conduct of the study, 
consultations with each of the subjects were 
done. Objectives, methodology and terms of 
reference specific to each of the three subjects 
were jointly worked out by the study team and 
the subjects. All agreements as regards the 
conduct of the study were finalized in these 
consultations. 


In Phase II, data gathering was conducted 
using structured and computer-based questi- 
onnaires. Questionnaires were based from 
initial data gathered from the subjects and 
were structured, based from the particularities 
of each of the three subjects. Informal discus- 
sions were also conducted. with some respond- 
ents, as the case may be. The use of computer- 
based questionnaires was to speed up analysis 
and firm up the basis for conclusions. 


A minimum of two weeks and maximum of 
one month was spent for integration in their 
specific areas of operation. Interviews were 
conducted during the integration. 


Data were processed after and validated 


through key persons of the subjects. Data gaps 


were filled in and controversial issues clarified. 
Preliminary analysis derived from trends in 
the data were shared with the subjects in this 
stage. 


In Phase III, which has not been achieved 
yet, results will be shared with other Helvetas 
Partners, comes March 20-26, 1988. 


E. Outcome 
The study-is just about to enter its Phase III. 
F. Follow-up actions 


Follow-ups will rest in the hands of Helvetas 
staff and management. Recommendations for- 
mulated by the study team will be subjected to 
further study by the subject and Helvetas. The 
role of the study team will end to the identifica- 
tion of problem areas where collaboration 
between the two organizations can be im- 
proved. 


G. Impact 


For the study team, this pioneering study on 
institutional planning and development adds 
to its knowledge and research experiences. To 
conduct a study on a burning issue regarding 
the interrelationship of NGOs and PBOs is a 
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contribution in itself to the whole Filipino 
struggle. It has widened every team member's 
perspective over the issue and has even incul- 
cated a want to go deeper into the realm of the 
issues. 


H. Issues in evaluation 


Along the process of the study, the study team 
was confronted with some realities which in 
the context of participatory evaluation will 
hamper the success of the approach. 


For one, Philippine NGOs are exposed to 
risks considering the impression the present 
and even the previous government has laid on 
them. Philippine NGOs espousing pro-people 
stance in their operation are suspected com- 
munist supporters and are, therefore, placed 
under tight surveillance and harassments. 


Under such an atmosphere, the staff and 
their beneficiaries tend to compartmentalize 
certain issues they think might put them in hot 
water if they divulge them to external groups. 
Consequently, the evaluators are left with no 
choice but to base their analysis and conclu- 
sions on “half-truth” or ‘“no-true” data. Hence, 
the objective of helping these people is for- 
feited. 


The case of heavy militarization in some 
areas where the study is being conducted 
makes the situation more difficult. 


In the case of Filipino peasants, the still 
prevailing neo-colonial culture evoked low- 
profile attitude among them. For these people 
who mostly did not finish formal schooling, 
‘evaluators or anybody of the middle-force 
nature are the more knowledgeable and that 
there are no rooms for their’ so-called petty 
ideas. This kind of attitude affects much the 
generation of data and their participation to 
the analysis of these data. 


The concept and practice of participatory 
evaluation was found useful and important 
among struggling nations like the Philippines. 
Empowerment is characterized by people’s 
participation. A criticism to any participatory 
activity, however, is its expensive nature. Any 
group conducting a participatory evaluation or 
any of that sort should have sufficient material 
and financial resources to achieve its goals. 


Integration with the target subjects is the 
key to the door of the realities surrounding 
them. It takes time and, therefore, money. The 
advent of scientific approaches to this type of 
activities adds to the expenses. 
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I. Background 


1.1 The Centro Latino Americano de Apoyo 


al Saber y la Educacion Popular (CLASEP)!, 
was founded in 1981 in Mexico City as a 
non-government organization. An edu- 
cational and non-profit organization, it 
is committed to assist in the strengthen- 
ing of the popular sector’s protagonistic 
capacity for the definition and attain- 
ment of a development that is in agree- 
ment with its interests and expectations. 


The Centre has mainly promoted the 
autosystematization and autoevaluation 
of the education of the masses. 


The CLASEP team has also promoted 
the establishment of the Programa Latino 
Americano de CEAAL? en Apoyo a la Sis- 
tematizacion y Autoevaluacion de Pro- 
gramas de Educacion Popular? and at a 
regional level, the Programa Global de 
Apoyo a la Sistematizacion y Autoeva- 
luacion de la Educacion Popular*—both 
of which are coordinated and counselled 
by CLASEP. 


The experiences resulting from these 
two programmes are contained in the 
case study presented in this paper. 


1.2 The lines of action that the Centre pro- 
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motes at present within the sphere of 
popular education, are: 


a. Systematization and autoevaluation; 
b. Participatory research/ action; 

c. People’s legal defence and culture; 
d. Training of farmers; 


e. Reconstruction ,of the history and 
struggles of popular sectors, 


f. Teaching of reading and writing. 


Latinamerican Centre for Popular Knowledge and 
Education | 
Latinamerican Centre for Adult Education 

CEAAL's Latinamerican Programme for the System- 
atization and Autoevaluation of Popular Education 
Programmes 3 

Global Prograrnme for the Systematization and Auto- 
evaluation of Popular Education 


1.3 The main types of services offered, are: 


a. Direct execution of programmes for 
popular education. 


b. Production of didactic material. 


c. Training of teachers for popular 
education. 


d. Diffusion. 
CLASEP is one of the founding members of 
CEAAL and of PRAXIS, an organization that 
groups 20 NGOs. 


II. Objectives of evaluation 


2.1 General objectives 


The purpose in both programmes is to 
strengthen the capacity of teachers as 
well as organizations, to consolidate their 
experiences in systematization and auto- 
evaluation processes, by providing pro- 
posals, methodologies and instruments 
that could help them reinforce the inten- 
tionality, efficacy, efficiency and impact 
of the work in popular education. 


2.2 Specific objectives 
Considering the principal methodologic 
aspects of the projects, the two pro- 
grammes intend to attain the following 
specific objectives: 


2.2.1 Prior to the projects’ execution: 


—Verify the consistency and coherence 
of its components. 

—Evaluate the project's viability. 

—Extend and consolidate the basis of 
group consent of the project. 

—Identify needs for education that may 
strengthen the response capacity of 
the promoting team to carry out the 
project. 


2.2.2 During the execution phase: 


—Contribute elements that improve 
the methodology and promotional 
practices developed by different sec- 
tors. 

—Specify and/or adapt intended goals 
and objectives, based on data obtained 
from the action’s development and on 
reflection thereon. 

—Validate or modify initial strategies. 

—Correct possible deviations. 

Identify needs for evaluation of the 
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viability of new actions. 
—Specify and adjust programming. 
—Identify needs for training in action. 
—Find out the response capacity of the 
promoting team to the project's re- 
quirements. | 


2.2.3 During the final phase: 


—Determine level of achievement with 
respect to proposed goals and objec- 
tives. 

—Evaluate the impact of actions carried 
out on sought-after effects as well as 
on unexpected results. 

—Portray the obstacles, favourable as- 
pects, successes and failures of the 
process followed. 

—Identify other projects or actions 
consensual to the executed project. 


III. Main principles operationalized 
during the process and focus of parti- 
cipatory evaluation 


With the development of the obtained metho- 
dology, the validation of the following princi- 
ples is sought: 


a. Significance and relavancy. 
b. Articulated reading of reality. 
Globality. 


c 
d. Historicity. 


e. Socialization. 

f. Relativity. 

g.  Criticability. 

h. Dynamic recurrence.. 


Self management participation. 
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IV. Methodology and process of parti- 
cipatory evaluation 


As one of the outcomes of the programme, a 
basic methodologic. scheme has been deve- 
loped that is considered valid for all evaluation 
"processes, and it is integrated by seven phases/ 
elements: 
1. Characterization of all questions to be 
answered by evaluation. 


2.  Systematization of actions, facts and fields 
referred to by the questions. 


3. Create or-adopt the reference framework 
or parameter. 


Establish contrast between 2 and 3. 
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5. Design a scale to express the results in 
magnitudes when required. 


6. Establish conclusions or judgements re- 
sulting from the contrast in phase 4. 


7. Devise the products/instruments that 
make applicable the results of the evalua- 
tion process and that can be applied to 
the project’s praxis. 


In order to instrument the global evaluation 
phases of a project, create an instrument that 
allows capture of the dynamics of this type of 
projects, integrated by the following elements: 


a. Characterization and nature of the. 
project. 


b. Ideology (principles and objectives). 
Methodological strategy. 
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Global historical and social context. 


e. Local context. 
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Characterization of all members of the 
organization, community or social group 
and its history. 


g. Characterization of the promoting team. 
h. Institutional context. 


i. Instrumentation and development of the 
project. 


j. Results and impact. 


CARLOS CADENA 


Loga{OUd FHL JO TUN.LYN 
CNV NOLLVZIMSLOVUVHD | 


NOLLVZLLVWH.LSAS WOU ANAHDS OIDOTOCGOH.LAW OISVd AHL 
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SARVODAYA RESEARCH 


INSTITUTE, Sri Lanka 


Sarvodaya Shramadana Movement was started 
in 1958, with community development activi- 
ties which was done by a group of volunteers. It 
was created to motivate the rural people to 
take part in their own development. Sarvodaya 
Shtamadana Movement later established a 
research wing to facilitate these activities. Sar- 
vodaya Research Institute in which I worked as 
a researcher from 1978 to 1982.was interested 
in participatory research, with the specific idea 
securing the people’s participation at the 
maximum level. It was accepted that without 
involving people at the research level no par- 
ticipation of the people in Sarvodaya activities 
could be expected. At that time Professor Nad- 
asena Ratnapala who was the Director of Sar- 
vodaya Research Institute and who had much 
experience in the field of participatory research 
was concerned that SRI researchers should 
employ this method in order to involve peo- 
ple’s participation. Professor Nandasena Rat- 
napala has done a good number of participa- 
tory research work such as The Beggar in Sri 
Lanka, Community Participation in Voluntary 
Organization, Police Public Relations, Alcoho- 
lism in a Sinhalese Village, etc. 


This evaluation took place ina village called 
Uragasmanhandiya in southern Sri Lanka. 
One enthusiastic Public Health Instructor 
approached us through Sarvodaya Shrama- 
dana Movement. He was seeking help to solve 
problems in his own area where he has been 
living since his birth and has worked as a PHI 
for 10 years. Working as a PHI, he has expe- 
rienced a lot of socio-economic problems in his 
own area, and through this experience he 
knew that the problems could not be solved 
without the participation of the people. 


He wanted to find out how he could moti- 
vate people to identify their own needs and 
seek their own solution for them. It is in this 
context that we evolved this strategy from our 

own experiences in the field. 


The objective ot the evaluation was to iden- 
tify the basic needs of the people and device 
strategies according to their likes and dislikes 
and involve them in translating these strate- 
gies into practice together with self evaluation. 


Our emphasis was on getting at the decision 
makers in these villages. In order to do this, 
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young people (both men and women) from the 
villages were motivated to identify such 
groups. They identified one mother, two young 
men and women and three elderly traditional 
leaders such as the village Buddhist monks, 
ayurvedic physician and village teacher. 


Accordingly three groups came together in 
various places and talked about their needs. 
We as researchers participated in their discus- 
sions restricting our participation to the min- 
imum level. Our idea was to make them talk 
and identify their needs without our inter- 
vention. 


From these groups emerged their own plans 
for meeting their needs. Children’s health 
including nutrition was a problem in these 
villages. As a consequence mother’s group in 
each village came into being. In addition to 
this, young people’s groups were organized. 
Traditional leaders and the young leaderswere 
brought together. When they were in conflict 
we as researchers helped them to resolve them 
not by forcing over solution, but by gradually 
enabling them to understand each other. 
Groups met regularly and evaluated their own 
impact focussing attention on their weak- 
nesses. / 


Methodology adopted in this exercise con- 
sisted of the foilowing: 


1. Living with the people for considerable 
time and becoming one of them. This 
was possible by the presence of research 
workers of the village by our long associ- 
ation with them. 


2. By taking part in common development 
activities such as Shramadanas, mater- . 
nity and child clinics etc. 


3. Cultural activities based on the village 
temple. 


4. Educational activities, organizations of 
pre-schools etc. 


5. Group discussions with the minimum 
participation of the researchers. 


6. Encouragement of exchanges of ideas 
pertaining to this evaluation on all in- 
formal occasions such as women go for 
water gather together in their workpla- 
ces, and go to maternity clinics. 


The outcome was the emergence of felt 
needs such as the ill health of little children, 
lack of nutritional facilities for children and 
mothers, etc. These needs were answered by 
the people, by their eagerness to start pre 
schools in every village; community kitchens 
latrines and soliciting the people’s help and 
the governmental help for better health facili- 
ties. There was a continuous evaluation as 
these activities were initiated. 


Whenever possible follow up action was 
carried out in order to strengthen the pro- 
gramme. Shortcomings were identified and 
the people themsleves with their own resour- 
ces attempted to meet these shortcomings. 


Sarvodaya Research Institute and myself 
gathered from this a number of insights, which 
were later used involving community partici- 
pation. 


The most important issue was to raise the 
consciousness of the people that could make 
them identify shortcomings in their own pro- 
grammes. Various strategies have to be devised 
in order to raise their consciousness to this 
level. As the villages, we selected, were in the 
majority very poor, socially extremely deprived 
with only opportunities for marginal educa- 
tion, this key issue was a perennial problem for 
-uS. 


But anyhow since there are other villages 
who do possess iderftical characteristics. The 
experience we are gathered here enabled us to 
refine these programmes subsequently. How 
could poor, uneducated socially backward 
communities be motivated to raise their con- 
sciousness to a level that would make them 
perceive their own problems and strategies to 
solve them and also become their own critics? 


WIMALASENA PITIGALA 


6] 


MAJGAON MARGINAL 
FARMERS SAMITY, Bangla Desh 


1. History of the organization 


I selected for Participatory Evaluation (PE) a 
project on re-excavation and enlargement of a 
tank for modern fish culture by members of a 
marginal farmers’ organized group under Cari- 
tas-DEEDS programme. The group was founded 
20 May 1982 as the Majgaon Marginal Farmers 
Samity, Uttar Para, Rajshahi District. Twenty- 
three members, many of them landless, formed 
the original membership. Only one member 
left the group (in January 1988) and 13 of the 
members are very faithful in attending all the 
meetings. The largest size of landholding is 1.4 
acres. Three members were educated to Class 
V and can read and write while the secretary 
had passed the Madrassa high school. 


In early 1984 the group excavated a tank 
under the Rural Works Programme (RWP) of 
Caritas, greatly enlarging a small tank. Caritas 
expended Tk 31,550.19 for this work and the 
members cut 10 per cent of the earth as self- 
help contribution (the amount was reduced 
because the earth was hard). All the samity 
members worked on the project, besides other 


landless labourers. They began fish culture the 


same year under the Caritas Aquaculture 
Development Programme (ADP), stocking the 
traditional “Big 3” (rui, catla and mrigal), plus 
silver, grass and mirror carp. There were no 
profits in 1985 when the tank was first fished. 
In 1986 the profit was substantial but was not 
distributed among the members. According to 
a group decision those who did the work were 
to get a double share, while ordinary members 
would get a single share. 


The group had carried out farming pfe- 
viously on their small pieces of land; three 
acres was accepted by Caritas as their under- 
standing of a marginal farmer (though 
government considers a marginal farmer to 
own two acres or less, since 2.5 acres is the 
average size farm in Bangla Desh. 


During the evaluation of the RWP in 1984, 
we visited the tank while the excavation was 
going on. In addition to the official report on 
the evaluation, I also prepared a justice analy- 
sis of several projects we had observed in order 
that the regional offices might discuss the pro- 
ject in this light and see if changes were 
needed. 
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I pointed out that this group was already 
doing IRRI cultivation in winter under a BRDP 
deep tubewell, for which all members had 
received loans of Tk 2000. The landholdings of 
the samity members had not been listed. It 
seemed to me that this samity was already 
much better situated economically than most 
of the hundreds of Caritas groups of the land- — 
less. Therefore, why couldn't the fishery pro- 
ject be given toa landless group? I never hada 
reply from the Regional Office and do not 
know if the case was ever discussed. 


The Caritas samities were all to do PE as 
part of the annual evaluation process. I en- 
quired about whether such evaluations had 
taken place or not. Since the samity has been 
going under Caritas direction for five years, it 
should be ready for independent self-reliant 
development with technical help when needed 
on the fishery side. Therefore, they should be 
able to do PE by now. They were allowed to do 
unguided evaluation on their own as the first 
step. The project Managing Committee also 
did the same. Thus, the ability of both groups 
to do evaluation on their own without outside 
help or facilitation was first tested. 


The second step was to do PE with the 
group with the aid of a facilitator to see if the 


group itself could identify weaknesses in eva- 


luation process. 
2. Objectives 


a. Tolearn the present level of the group's. 
ability to understand the aims of evalua- 
tion, the methods of evaluation and the 
actual results of evaluation; 


b.  Toascertain this without influencing the — 
group in any way, ie., asking them | 
simply to evaluate their experience, how 
they understand what they are doing and 
how they feel about tt; 


c. To analyse the results of the different | 
group evaluations and then to sit with 
each group for an open-ended discussion 
to learn their ideas and plans about 
changes on improvements in the project; 


d. To test hypotheses about the principles ) 
involved; 


To compare the participatory process 
without an outside facilitator to that with 
a facilitator; 


To see what a trained observer can add to 
what has been learned through the partici- 
ipatory process; 


To see the effect of an outsider, an expe- 


rienced participant/observer, on the PE 
process. 


3. Main principles operationalized 
(hypotheses in advance) 


Hypotheses were set in advance,-which, if veri- 
fied, would generate the main principles oper- 
ationalized during the PE process. Each hypo- 
thesis and the reality observed are discussed 
together in the section on outcome (No.6). 


a. 


b. 


e. 


4, 


a. 


The project people know their needs and 
understand their problems best. 


The. group was sufficiently organized 
and trained in advanee to be able to carry 
out the project themselves. 


The group members understand suffi- 
ciently about the meaning of evaluation 
and the means to be used. 


There is a high degree of trust and coop- 
eration manifested among the partners. 


A trained evaluator and facilitator can 
supplement the results of PE substantially. 


However, changes and decisions made by 
the group itself are mote effective. 
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Focus of evaluation 


Organization — group cohesion; group 
understanding of project objectives. 


Activity — understanding of technical 
processes, sharing of work and produc- 
tion, understanding of project results. 


Impact — economic benefit to each 
member, growth in social power and in- 
fluence, ability to resist or counteract 
local injustices. 

Change — changes in: group attitudes in 
the local society or social system, in bal- 
ance of power, in attitudes of non-group 
members, etc. 


5. Methodology 


i 


The group was asked through the regional 
office to evaluate its own activities with- 


out influencing them in any way in order 
to see their present ability to understand 
the aims and the methods of evaluation 
and to -analyse the actual results of 
evaluation. 


2. = fhe project Managing Committee was | 


asked to discuss and evaluate the project 
and later to analyse the report of the 
Samity members. 


3. The Regional. Office was then asked to 


have their project committee study the 
two reports and point out the weaknesses 
and what further steps should have been 
taken and could still be taken. 


4. The outside facilitator (myself) then ana- 


lysed the three written reports and sat 
with all the three gr6ups to question 
them in order to bring out further aspects 
of the evaluation which had not been 
raised at all or had not been expressed 
adequately and to learn their ideas and. 
plans for improvements in the project. 


5. The outside facilitator then made inde- 


pendent enquiries to try to corroborate 
the PE given by the groups. 


6. The facilitator tested the hypotheses 


formulated under No. 3 against the var- 
ious evaluations to see if they had been 
verified. 


6. Outcome of evaluation 


a. Group members: 


Their chief hopes were to: cut and improve the 
tank, earn something digging it, improve the 
samity by fish culture, cut other tanks in the 
future from their profits, use the water, learn 
modern methods of fish culture, cultivate the 
banks, build up samity funds, help the poor of 
their village. 


Their failures were: loss through flooding, 
improper feeding, bad relations with the owner 
and his failure to observe the contract. The 
reasons for failure were: negligence of the 
manager and chairman and their failure to 
attend the weekly meetings, their taking most 
of the profits, failure to feed the fish in spite of 
money in their fund. 


Their successes were: their water problem 
was solved, they made money in cutting the 
tank, they learnt modern fish culture, they 
helped the poor. 
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b. Local Project Managing Committee 


They had basically the same points but elabo- 
rated them more. They blamed the failures on 
the negligence of the ordinary members and 
their failure to attend meetings and pay their 
weekly savings. They paid for the fish food 
from their own pockets and undertook all the 
responsibility for the pond. They also said that 
they couldn’t extract fish regularly due to lack 
of a net. Some members got loans from the 
samity fund under regulation procedures but 
they didn’t report on repayments. They main- 
tained that they got no more improvement and 
benefit than the ordinary members. 


c. Caritas Regional Project Committee 


The Regional Project Committee at Rajshahi 
evaluated the two reports of the samity mem- 
bers and the project Managing Committee and 
added their own comments concerning signifi- 
cant omissions. They.stated that the project 
was begun with great eagerness and that the 
members had much training and were suffi- 
ciently motivated. The excavation work under 
the RWP had been very successful; all the 
members had taken part and had boosted their 
income and savings through the project. 


There was every hope that the fish culture 
project would also be successful. In fact, the 
group ran well up to 1986 and some poor 
landless moved up to the middle class. How- 
ever, in 1987 Caritas involvement lessened due 
to the group’s inclusion in the final or Phase II 
of the DEEDS programme (concentrating on 
adult literacy). After Phase III, phase out 
would occur. 


A leadership conflict arose because the 
educated secretary, who also acted as treasurer, 
had used Tk 2800 of the cash holdings for his 
own purposes. The members knew it and 
brought pressure on him but he has still not 
repaid inspite of promises. 


Caritas’ loans of Tk 10,000 were taken by 10 
members in 1986 anda samity loan of Tk 6000 
was given to six members (two got from both 
sources) for IRRI rice cultivation. But succes- 
sive crop failures at harvest time, due to tor- 
rential rains in 1986 and floods in 1987, pre- 
vented their repaying the loans. 


Moreover, the owner of the tank did not live 
up to the terms of the contract and did not 
allow the members to plant vegetables on the 
banks. He destroyed their banana plantation 
on the banks and grew grass and coconuts for 
himself. According to the contract, he was to 
receive 25 per cent of the profits from the 
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banks. This impasse continues to the present. 


The group is now disorganized but the land- 
less members would like to separate from the 
marginal farmers (after settlement of all loans 
and repayment by the secretary) and manage 
the fish tank themselves. They feel that the 
marginal farmers got much benefit through 
their agriculture loans but they didn’t get any 
loan or benefit. 


d. Final group evaluation 


An evaluation session was held with the sam- 
ity and Managing Committee members 
together on 21 January 1988. The Caritas per- 
sonnel-had stated that there was no domina- 
tion in the samity and that even the poorest 
landless would speak up freely in front of the 
Managing Committee. All attended except the 
secretary, who was away on urgent business 
(one of the landless used this as an example of 
how the Managing Committee were too busy 
to give enough attention to the fish culture). I 
questioned the members in order to obtain 
more information and they spoke out freely - 
and intelligently about all their problems. 
They all knew the facts about their samity: 
how much was deposited in the bank, how 
many fingerlings were put in the pond, how 
many fish were caught and how much money 
was made, etc. 


The main point was that the landless have 
come to realize that their own interests were 
not being served by the marginal farmers, 
especially the Managing Committee. The mar- 
ginal farmers got substantial loans while they 
did not. It was for this very reason that Caritas 
had begun to work only with homogeneous 
groups, but this group had organized itself and 
presented itself to Caritas, so the landless and 
marginal farmers were not separated. The 
landless now want to go their own way and feel 
that they are sufficiently cohesive and trained 
to work well together. 


7. Verification of hypotheses (No. 3) 


a. The group had sufficient training and 
b. self-awareness to evaluate the project 


_¢. themselves without outside help (though 


it was the first time they were making 
such an evaluation). They understood 
clearly the essential issues and problems 
and expressed them sufficiently. 


d. The trust among members broke down 
in 1986, after the secretary was disco- 
vered using funds for himself. 


e. The outside évaluator (participanc- 
observer) added nothing to the process 
of evaluation except to clarify some of 
the points which were too briefly expres- 
sed on paper and to get more details 
about them. His presence at the final 
evaluation did not impede the flow of 
information and the landless members 
openly criticized the Managing Commit- 
tee and realistically exposed their hopes 
and plans for the future. 


The Caritas Project Committee added 
helpful information but except 
for their explanation about loans to the 
members none of their other comments 
was indispensable for a clear understand- 
ing of the project. 


8. Impact of evaluation 


As a result of the evaluation, the samity and 
Caritas understood clearly that the main prob- 
lem of the group was the disparity between the 
marginal farmers and the landless. Both groups 
appreciated the need of separating from each 
other. All appreciated what help Caritas had 
given them but understood that they had to 
work out the solution to this big problem b 
themselves. 


The impact on me personally was tremend- 
ous, since my previous extensive contacts with 
field groups of the landless showed them to be, 
for the most part, willing to be led and placing 
their confidence in the leaders of. their group. 
The present groups of landless were incisive 
and logical in their criticism.of what was 
wrong with their samity. 


9. Issues in evaluation 


a. Are individuals and groups sufficiently 
able to undertake PE without special or 
scientific training? 

b. Should the PE itself be the training pro- 
cess or should some prior instruction be 
given during regular group meetings? 

c. Should changes in plans and projects be 
initiated at the time of PE or independent 
of it? 

d. Does a trained and experienced evalua- 
tor and facilitator add substantially to the 
PE process?’ 

e. Does an outside facilitator impede the 
process of self-reliant development in 


the long run? RW TIMM 
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‘LUISA AMANDA 


ESPINOZA WOMEN’S MOVEMENT, 


Nicaragua 


National context 


On July 19, 1979, the Sandinista Popular Revo- 
lution triumphed in Nicaragua after 45 years 
of oppression by the Somocista dictatorship. 
This revolution is the culmination of the 
heroic initiative of General Sandino who 
fought US military intervention in the country 
between 1927 and 1933, forcing the United 
States to withdraw. Later Sandino was vilely 
assassinated by the forces of Anastasio Somoza 
Garcia, inheritor of that US occupation. , 


The Sandinista Nationa! Liberation Front 
led a broad alliance of economic, social and 
political forces against the Somocista military 
dictatorship. It formed a broad coalition 
government that carried out democratic, 
nationalist and popular tasks such as the 
nationalization of the banks and foreign com- 
merce, the expansion of education and health 
for the popular sectors, the unionization of 


workers and peasants and the initiation of a 


democratic agrarian reform in favour of the 
peasantry. At the same time, it developed a 
foreign policy of non-alignment, and joined 
the Movement of Non-aligned Countries. 


With the coming to office of the Reagan 
Administration in 1981, a systematic cam- 
paign against the Sandinista Popular Revolu- 
tion was initiated that obliged it to redouble its 
military efforts to confront the mercenary 
bands that have caused 50,000 victims to date 
and to counteract the serious effects of the US 
economic blockade and financial aggression 
against the weak Nicaraguan economy. 


As part of the democratization process, the 
first free elections in the history of the country 
were held in 1984, and in 1987 the political 
constitution of the country was promulgated, 
encompassing the main democratic, national- 
ist and popular principles of the revolution. In 
1988 this democratization process is continu- 


ing with the upcoming elections for the Cen-: 


tral American Parliament and for municipal 
government. It is appropriate to point out that 
in this electoral process and civic political 
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struggle, a total of 15 political parties are func- 
tioning, seven of which are represented in the 
parliament. | 


The popular sectors have advanced palpably 
in their participatory, organizational, union 
and mobilizing capacity. Hundreds of unions 
of industrial and rural workers have been 
formed in the countryside and the cities; health 
and education workers have organized, as have 
professionals, residents of poor neighbour- 
hoods, youth and the Luisa Amanda Espinoza 
Women’s Movement. 


In this way, the Sandinista revolution ts sus- 
tained by a popular movement that is becom- 
ing for the first time in history a pillar of the 
participatory democracy we are building; it is 
becoming a free and organized expression of 
the people, just as the armed forces and popu- 
lar militias are the expression of the armed 
people, the principal guarantee of the suste- 
nance of the revolution in the face of the 
aggressiveness of the main world power, which 
openly, through its Administration and its 
Congress, sustains the mercenary aggression. 


It should be pointed out that these mercen- 
ary forces are in a progressive military decline, 
the product of the offensive capacity of the 
Nicaraguan people. Nonetheless, the successes 
of the Reagan Administration can be seen in 


the physical destruction of cooperatives, school 


centres, workplaces, etc., and in the deteriora- 
tion of the economic situation of the country in 
general, which has forced the development of 
an economic survival scheme that will un- 
doubtedly last for many years. 


Nicaragua, together with other Central 
American countries and with the support of 
Latin America and other democratic forces of — 
the world, is promoting a peace process in the 
Central American region, based on the Esqui- 
pulas Accords signed in August 1987 and rati- 
fied in January of this year by the Central. 
American presidents. Despite broad interna- 
tional support for the Esquipulas initiative, the 
US administration is trying fo sabotage it and 
continue its aggressive policy in the region. 


It is in this context that the Luisa Amanda 
Espinoza Women’s Movement is promoting 
the pilot experiment of the Training Pro- 
gramme for Base Promotors and Leaders in 
Region IV of the country, an old coffee grow- 
ing region and an important population centre 
since the precolonial period. It is the zone of 
the country with the highest population den- 
sity despite its small size—only four per cent 
of the national territory, 


The Luisa Amanda Espinoza Women’s 
Movement 


The Luisa Amanda Espinoza Women’s 
Movement (AMNLAE) emerged with the revo- 
lutionary triumph on July 19, 1979, as a result 
of the massive participation of women in the 
struggle against Somocisme and the political 
will of the FSLN to support and promote the 
diverse mass organizations mentioned above 
so they could represent and struggle for the 
particular interests of various social sectors in 
conjunction with the overall interests of the 
Sandinista Popular Revolution. | 


The immediate antecedent of AMNLAE was 
AMPRONAC (Association of Women Facing 
the National Problematic), founded at the end 
of 1977 at the initiative of the Sandinista Front 
and based in the struggle for human rights 
during the Somocista regime. 


In March of last year the FSLN presented its 
Proclamation for work with Nicaraguan wo- 
men, which marks a new stage in AMNLAE’s 
work. Previously, we had some experiences 
around the resolution of specific women’s 
problems, but AMNLAE’s activity was concen- 
trated fundamentally in promoting women’s 
participation in the general tasks of the 
revolution. 


The Proclamation emphasises AMNLAE as a 
fundamental movement for the ideological 
transformation of men and women, based on 
the resolution of the specific problems of 
women, which permits them to advance in 
achieving their emancipation and in greater 


and better participation in the general tasks of 


the revolution. 


In this framework, political-ideological work 
is converted into the main focal point of 
AMNLAE, to be expressed in the different tools 
of propaganda,training and research which, 
although they have their specificities, must be 
managed 1n an articulated, parallel and com- 
plementary way around the organization of 
the movement and its platform of struggle. 


Before the Proclamation AMNLAE's dynamic 
for political-ideological work was mainly cen- 
tered in sporadic propaganda activities for 
campaigns, such as the preparation of leaflets, 
radio spots, some pamphlets, etc. 


Few training activities were carried out 
either at the central level or other levels of 
AMNLAE. Those carried out before the Proc- 
lamation were around general concepts of 
propaganda, handling of cameras and tape 
recorders, popular education techniques, legal 
aspects and sex education, among others. In 
turn, such actions were not framed within a 
training strategy with either methodologies or 
explicit techniques. 


Now AMNLAE is promoting the execution 
of various workshops to study the FSLN Proc- 
lamation. In turn, the Training Section pres- 
ented a guide document for training ‘that 
included the development of a series of activi- 
ties with the Executive Secretariat and the 
National Committee, permitting the prepara- 
tion of the Movement’s Platform of Struggle 
and its new Political-Organizational Concep- 
tion. At the same time, the pilot experiment of 
a Training Programme for Base Leaders and 
Promotors of the movement was begun last 
September in Region IV, with the goal of exe- 
cuting it in the rest of the regions this year. 


With. respect to the field of research, 
AMNLAE is beginning to coordinate with the 
Women’s Institute, the Agrarian Reform Re- 
search Centre and the Nicaraguan Institute of 
Social Research, particularly to form a support 
and advisory team for the Training Programme 
for Base Leaders and Promoters based on 
participatory-action research methodology. 


The Training Programme for Base 
Leaders and Promoters of AMNLAE 


The new political-organizational conception 
of the broad, democratic, plurisectoral Move- 
ment, which is inserted with different rhythms 
into the various mass organizations, trade 
unions, etc., is beginning to eliminate the 
previous form of organizing work, which was 
as an organization parallel to these others. 


It is indispensable and urgent to place 
promotors or base professionals from the 
movement in the various sectors where they 
work to find and train base leaders, learn about 
and jointly systematize the problems of women 
in their work areas, and seek priority solutions 
that permit qualitatively and quantitatively 
greater participation in the major revolutionary 
tasks. 
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It is- equally important to promote the 
exchange of experiences among women base 
leaders who have developed a certain level of 
awareness about detecting and solving the 
specific gender problematic. 


General objective of the programme 


To contribute to the conduction of the move- 
ment through training in work organizing and 
planning. 


Specific objectives 


That the participants : 


1. intesnalize the Platform of Struggle and 
the new organizational-political con- 
ception of the movement; 


2. comprehend their role within the move- 
ment, unifying concepts and criteria of 
the work;. 


3. develop awareness of their condition as 
women, their specific problematic and 
the factors that favour or impede their 
Participation in revolutionary tasks; 


4. acquire basic knowledge about popular 
education; 


5. develop participatory-action investiga- 


tions that permit them to do diagnostics 
of their work areas; 


6. prepare work and struggle plans for 
their respective work centres; 


7. become familiar with some basic training 
demands. 


Methodology used 


We proposed developing a participatory-action 
investigation that would quickly permit base 
leaders and promoters to do diagnostics of 
their work areas in relation to the socio- 
economic situation prevailing there, the 
- organizational-political situation of the union 
or sectoral body and the specific problematic of 
women. 


For that purpose, a Support Team, made up 
of researchers working in the various research 
centres of the country, prepared a proposal for 
instruments which was_ presented to the 
participants in a first intersectoral workshop 
we developed, employing the popular education 
methodology. This was used in the remainder 
of the activities. 
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Afterward, we proposed providing follow- 
up and support to the execution of the diag- 
nostic to later systematize the information 
obtained in such a way as to permit its analysis. 
In sector meetings orientations were given for 
this systematization. 


In sectoral workshops the data was reflected 
on and the respective struggle plans were 
prepared. 


_ Finally, in a third intersectoral workshop 
the struggle plans were socialized and metho- 
dological and technical guidelines were set up 
for the preparation for the work plans. 


Participants 


In addition to the sectoral base leaders and 
professional promotors of AMNLAE in Region 
IV, we proposed that the following participate 
in the varieus activities of the programme: 


—the regional executive secretery of 
AMNLAE, and particularly its secretary 
general and person in charge of pro- 
paganda and training; 


—the sectoral delegates of the Regional 
Committee of the peasant, industrial 
worker and community sectors; 


—those in charge of members of the 
training teams of the sectors participating 
in the programme ; 


—those in charge of AMNLAE training 
from the other regions to permit them to 
function as trainers in action; 


—the support team for the training pro- 
gramme, made up of the various resear- 
chers from the research centres and 
members of the sectoral training teams; 
and 


—AMNLAE's national training team. 


Programme phases 


This pilot experiment has implied the follow- 
ing phases : 


First Phase : Sensitizing the 
participants 


In this phase, the following actions were 
carried out : 


-—presentation and approval of the pro- 
gramme of AMNLAE’s National Com- 
mittee and of the Executive Secretariat of 
Region IV; 


—a working meeting in the region with the 
Regional Executive to carry out a series 
_of tasks such as promoting study of the 
proclamation and the organisational-poli- 
tical conception of the movement among 
programme participants; placing the pro- 
moters in their work sectors; collecting 
and studying information from the sec- 
tor where they were placed ; presenta- 
tion, by the participants, of the list of 
information considered important to in- 
clude in the instruments for the diagnos- 
tics; presentation of official list of partic- 
ipants at the first workshop. 


—calling the first workshop of the 
programme 


Second Phase : Participatory diagnos- 
tic by region 


In this phase the following actions were car- 
ried out : 


—a three-day workshop was held with the 
participation of 37 companeras with the 
goal of accomplishing the following 
objectives: 


1. The participants characterized their 
previous work and gained an under- 
standing of the role they are going to 
assume. 


2. They acquired the knowledge and 
skills necessary to do a participatory 
diagnostic of their respective work 
areas and prepare their correspond- 
ing work plans for its realization. 


3. They became farhiliar with some 
theoretical elements for observing 
and analysing the reality of Nicara- 
guan women. 


—they carried out 4 participatory diag- 
nostic through which a total of 2303 
people in the base participated: 1171 
women from poor neighbourhoods, 395 
‘ndustrial workers and 837 peasants, 
both male and female. It affected 66 
neighbourhoods, nine factories, 21 rural 
cooperative development polls and 33 
population areas in the countryside. 


—working meetings; by sector, oriented 
people about how to systemative the 
information. 


Third Phase : Preparation and execu: 
tion of the struggle and work plans 


This phase has implied the following actions: 


—A workshop by sector (three in all) to 
analyse the information and prepare the 
respective struggle plans of industrial 
workers, peasants and neighbourhood 
dwellers from a gender perspective. 


—Evaluation of the development of the 
programme to date, which was carried 
out in the intersectoral workshop menti- 
oned above. 


—An intersectoral workshop to socialize 
the struggle plans corresponding to each 
of the participating sectors and to become 
familiar with some technical-methodo- 
logical orientations for preparing their 
work plans. 


—A process of executing the work plans by 
the promoters and base leaders. 


Fourth Phase : Follow-up, control and 
evaluation 


We sought to prepare a system of follow-up, 
control and evaluation of the work plans in 


support of the work of leading the movement. 


Evaluation of the programme 


Although we understood the evaluation as a 
permanent and continuing process in all stages 
of the training programmes (diagnostic or 
detection of necessities, planning, carrying out, 
and follow-up and feedback), it was not con- 
ceived of that way for the programme for 
movement leaders. That was mainly due to a 
lack of sufficient knowledge and personnel in 
the training team for the design and execution 


of the system. 


Nonetheless, we carried out some partial 
evaluations in the stage of effecting the pro- 
gramme. Although they were limited and we 
still have to continue processing them, they 
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"can serve in some measure to make the varia- 
tions and adjustments necessary to execute the 
programme in the other regions of the coun- 
try, as well as to give adequate follow-up to'this 
pilot experiment. 


Below we present the most important results 
of these, including the objectives that we set 
out and the techniques we used. 


Results 


We did two partial evaluations. The first dur- 
ing the second phase of the programme, in the 
first intersectoral workshop; the second in the 
third phase of the programme, in the second 
intersectoral workshop. 


In both evaluations we implicitly proposed 
the following general objectives: 


—to verify whether the actions were éxe- 
cuted in accord with the programmed 
activities and according to the defined 
principles and procedures, detecting devia- 
tions so as to make the appropriate 
corrections; 


—to measure the efficiency shown during 
the execution of the training, as. well as 
the effectiveness of the methods and 
instruments employed; 


—to measure efficacy in achieving the pro- 
grammed objectives and goals, identify- 
ing the results of the training actions. 


Evaluation carried out in the first 
workshop 


In this evaluation we proposed the following 
specific objectives: 


—to learn about the experience of political- 
organizational work that the trainees 
have; 


—to verify the level of knowledge of the 
trainees with respect to the gender proble- 
matic; 


—to assess the fulfilment of the objectives 
of the workshop and the limits that up to 
now were showing up in the programme. 


To obtain the required information we used 
the following instruments: 


—An initial questionnaire on the Efficacy 
of the Teaching-Learning Process, pres- 
ented at the beginning of the workshop, 
with a series of questions about related 
aspects of leadership and organization of 
the movement, as well as the specific 
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problematic of women. They were filled 
out individually. 


—A Technical-Operative Questionnaire, 
with group questions and responses, 
aimed at evaluating the workshop in _ 
relation to fulfilment of organizational 
objectives and aspects and their coor- - 
dination. 


—A Final Individual Questionnaire on the 
Efficacy of the Teaching-Learning Pro- 
cess to contrast it with the initial knowl- 
edge of the workshop. 


—Direct observation of the process. 


The results of the Final Questionnaire on 
Efficacy were the following: 

—There was consensus regarding the per- 
sonal characteristics that leaders must 
develop, which are to: be linked to the 
bases, be self-critical, start from reality to 
carry out the work, be humble and use 
simple, communicative, planned and eva- — 
luative work styles and methods. 

—The companeras identified the most 
relevant manifestations of oppression 
suffered by women, which are physical 
mistreatment, economic dependency, the 
double day, little technical training, the 
reproduction of family roles as the pro- 
duct of a machista ideology, the existence 
of discriminatory laws and the low cultu- 
ral, political and ideological level, which 
carries with it submission and depen- 
dence. 


—The majority concluded that in order to 
begin to break the oppressive situation 
they are submitted to, Nicaraguan women 
have to acquire feminine consciousness, 
achieve higher levels of training, carry 
out various actions that permit ideological 
transformation, get the state institutions 
to be concerned with the rights of women 
and promote the preparation of laws in 
favour of women. Few mentioned their 
integration into economic life. 


—In relation to their comprehension about 
what a diagnostic is and what it is for, the 
majority responded that it was a study 
through which one acquired knowledge 
about reality and that it serves as a guide 
for the preparation of their work plans. 


—W ith respect to the aspects that must be 
contemplated in a work plan and their 
utility, the majority responded that these 
serve to organize the work and to direct 


it well. Some also pointed out their use- 
fulness for planning. 


Regarding the -technical-operative questi- 
onnaire, which allowed them to evaluate the 
workshop, the majority answered as follows: 


—The objectives of acquiring knowledge 
for carrying out the diagnostic and com- 
prehension about the situation of oppres- 
sion that women live under were ful- 
filled; and the differentiation of general 
demands: of women and their specific 
gender demands was accomplished. 


—In relation to the methodology of the 
workshop, they concluded that this per- 
mitted them to fix the knowledge in 
their minds and promote the participa- 
tion of the leaders, in particular through 
the group work; that the coordination 
had been very good, as well as the mate- 
rial conditions necessary for housing, 
food and development of the taller. 


We have not yet systematized the Initial 
Questionnaire on the Efficacy of the Educa- 
tion-Learning Process and therefore it was not 
contrasted with the similar Final Question- 
naire. 


Limitations 


Through direct observation at the workshop 
and the analysis of the above-mentioned eva- 
luations, we were able to detect the following 
limitations: 


—At that moment, the programme was not 
taken up by the majority of the various 
leadership bodies of the movement, par- 
ticularly in the region itself, which under- 
mined the coordination necessary for its 
implementation. The importance of 
training for and in function of the organi- 
zation had not been assimilated, which 
meant that attendance at the workshop 
was a little less than 50 per cent of those 
invited; in their majority promoters (90 
per cent) instead of base leaders, and the 
former were not all found in their work 
sectors, which was a fundamental requ- 
site for the workshop. 


This reflects the tension that exists 
between the old work styles and the 
assimilation of the forms we want to 
promote, which question the conjunctu- 
ralist, activist and disordered work styles. 
Furthermore, we need to be clear about 
the dialectic relation that exists between 
the movement, the vanguard and the 


State apparatus, and the way in which 
this should be expressed. Particularly, we 
need to better understand the level of 
autonomy that the movement should 
have to permit it to act democratically, 
with greater insistence, expressing 
women's different banners of struggle, 
and we need to develop greater convoca- 
tory capacity for the various activities 
carried out. 


—The less than 50 per cent attendance, 


mainly of promoters, implied the need 
for them to reproduce some issues of the 
workshop (such as the use of the diag- 
nostic instrument) without having been 
trained for it. 


—The instruments for the diagnostic were 


not sampled. It would have been desira- 
ble to do so for enriching both their lan- 
guage and content. Nonetheless, it started 
from the accumulated experience of the 
research members of the programmme’s 
support team. The ideal thing, as a con- 
sequence of the participatory methodol- 
ogy, would have been to hold a workshop 
strictly for the elaboration of these 
instruments. 


—The Support Team for AMNLAE Train- 


ing has not succeeded in coming together 
in a stable way with the research com- 
paneras of the different research centres 
and sectoral trainers. This implies the 
need for greater sensitization work about 
the potential of the participatory-action 
research methodology for organized, 
educational and research political action 
that precisely seeks to articulate knowl- 
edge acquired at the macro-academic 
level with popular knowledge. 


—It is necessary for the professionals and 


base leaders to go forward in an intense 
way in acquiring a feminist conscious- 
ness to enable them to develop greater 
analytic capacity in their work. Inasmuch 
as the programme contemplates the pre- 
sentation of theoretical themes in each of 
the activities proposed, it is necessary to 
execute the Intensive Programme about 
Gender Theory at the same time as this 
programme, to permit a more acceler- 
ated intellectual development by the 
companeras considering their leadership 
role and their need to coherently inter- 
pret the problematic they are facing in 
their daily work. 


Evaluation carried out in the second 
workshop 


In this evaluation we suggested, as a general 
objective, familiarizing ourselves in an overall 
way with the results to date of the whole pro- 
cess of training we are promoting, with the 
goal of deciding about continuing and repro- 
ducing it in the other regions of the country. 


We ‘used a question guide to respond in 
groups. This could be enriched by the partici- 
pants. In synthesis, the results were the follow- 


ing : 


—50 per cent of the participants in the 
second workshop were present in the 
first one. 


—The sectors were understanding the new 
political-organizational conception of the 
movement. 


—The diagnostic fulfilled its objective, 
permitting them greater knowledge 
about their work areas for preparing 
their respective struggle plans. Obstacles 
were also pointed out, such as lack of 
compliance with the programming that 
did not permit technical support during 
the information preparation phase, a 
weakness in the instruments that did not 
permit the questions to be correctly 
interpreted, the date to be quantified and 
thus greater knowledge of the sector to 
be gained; some were done by compane- 
ras who had not participated in the for- 
mation workshop for their implementa- 
tion. 


—The role of the promoters of the move- 
ment and their placement in the sectors 
was being defined. 


—The struggle plans of the sectors involved 
were prepared from a gender viewpoint. 


—The participants succeeded in differen- 
tiating general demands of women from 
their gender demands, as well as in better 
understanding the new participatory and 
democratic styles and methods that should 
characterize the work of the movement. 


—New base leaders were identified. 


—The work plans have been prepared for 
achieving the objectives proposed in the 
struggle plans. 


—Lack of support by the-top leadership 
bodies of the movement. 
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Reflection about the evaluative 
processes 


As a result of our experiment, its achievements 
and limitations as well as of various readings 
on the theme, we drew the following conclu- 
sions regarding the significance and impor- 
tance of the evaluative process. These will be 
taken up in the design and execution of evalua- 
tion systems that should complement the var- 
ious training experiences we promote, includ- 
ing follow-up of this pilot experiment and its 
reproduction in the other regions of the 
country. 


Ina very general way we understand as eva- 
luation the systematic process that identifies, 
explains and contributes information to sup- 
port the taking of decisions and feedback to the 
leaders and participants about proposals, 
actions or results of the programme to which it 
is applied. Furthermore, it permits, through 
appraisals and analysis, a comparison of the 
various elements of the programme with pre- 
viously determined parameters for integrating 
useful information at each moment in the tak- 
ing of decisions. 


We conceive it as an integral process of 
training, a system which must be present dur- 
ing all its moments or stages (diagnostics or 
detection of needs, planning, execution, 
follow-up and feedback); which is used as a 
method of research and knowledge to permit 
the detection, systematization and analysis of 
actions carried out with the goal of recovering 
and integrating experiences, giving feedback 
to their suggestions in a continuous way. 


In our pilot experiment we did some evalua- 
tions only in the execution stage of the pro- 
gramme. In future experiments, however, we 
are contemplating an evaluation system that 
includes all stages of the process. For example, 
we did not do an explicit diagnostic about 
training needs that we consider fundamental 
to being precise about the reach, depth and 
veracity of detecting needs, and which could be 
achieved by evaluating the sufficiency of the 
diagnostic. 

If, in the majority of cases, we situate the 
evaluation as a necessary moment within the 
realization of any event, we believe that it should 
not be understood as an aspect separated from 
the training actions, or as an external action that 
can be applied at moments and about aspects — 
isolated from the training process. It only makes 
sense if it is inserted in and directly related to this 
process, as an action tied to all training, as a 


research method for action, that it be present 
from start to finish. 


In the diagnostic and detection of needs, the 
evaluation permits us to determine if the objec- 
tives, content, strategies and structure of the 


programme generally respond to the previously 
detected needs. 


When the event is being planned, evalua- 
tion permits constant feedback about the pro- 
cess by analysing aspects such as the pro- 
gramme’'s coherency and the viability of the 
objectives, techniques, etc., proposed for its 
development. 


While carrying out the programme, evalua- 
tion can acquire greater dynamism by contri- 
buting elements that permit verification, con- 
trol and feedback about the training actions in 
function of the originally proposed objectives 
which had already been evaluated and deter- 
mined as appropriate to. respond to the 
detected needs. 


Finally, in the follow-up and feedback stage, 
evaluation, by taking the results of the differ- 
ent partial evaluations, permits us: an overall 
vision by detecting the successes and limita- 
tions to use in giving feedback about the devel- 
opment of future programmes: to evaluate the 
results of training insofar as it appraises the 
direct effects it had on the work of the trainees; 

as well as the contribution it made to the 
_ changes occurring in the region’s conditions. 


MALENA DEMONTIS 


TRAINING FOR 


PARTICIPATORY EVALUATION 


Introduction 


People’s participation has been strongly advo- 
cated for some time in the planning, implemen- 
tation and monitoring of a wide range of deve- 
lopment activities. Unfortunately, when it came 
to ‘evaluation’ the voices of the advocates of 
participation faltered. In fact, they usually pet- 
ered out. In the ensuing silence the fragments of 
their rationale gradually emerged to form a 
fairly coherent picture. It was not an attractive 
one. Its elements included an uncritical subser- 
vience to the mystiques of coriventional evalua- 
tion approaches, an unswerving bias towards 
complex, costly and highly quantitative evalua- 
tion methods, inability to perceive the need for a 
broader range of evaluation approaches and 
skills, and a general reluctance to extend partic- 
ipatory principles to evaluation. Such an atti- 
tude revealed a serious undet-rating particularly 
at middle and community level of the capacities 
of the practitioner actually involved in imple- 
menting development activities. It also revealed 
‘ vested interest groups. 


A popular keyword in framing objections to 
participatory evaluation has been the term 
‘scientific. Conventional evaluation methods 
were said to be more ‘scientific’. They were said 
to be more ‘objective’, reliable and valid. They 
were also said to generate a larger body of quan- 
titative data, supposedly more relevant to govern- 
ment agency needs. However, there has also 
been a slow but discernible and increasing dis- 
enchantment with the results of conventional 
evaluation methods. Thus moves have been 
made towards developing less costly, less expert- 
dependent and more self-reliant evaluation 
approaches. It is now argued that it is better to 
have less perfect but more usable data. It is also 
better to have less evaluation data which can 
more easily be shared than a massive amount 
which becomes the private (and often confiden- 

tial) possession of a few. 


Within this framework practitioners in dive- 
rse parts of the world have been developing a 
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range of evaluation approaches and methods ~ 
which are based on people’s participation. Much 
has been learnt and adapted from earlier parti- 
cipatory experiences in the fields of community 
development and adult education. People are 
involved in defining their own evaluation needs, 
building on existing intellectual and leadership 
capabilities and practical skills, and refining 
group work methods. Participants are enabled 
to analyse systematically their own reality and to 
increase self-reliance and self-determination. It 
is perhaps this last feature that is the most 
controversial. While many people in develop- 
ment activities may be ready to share responsi- 
bility, there are few who are genuinely ready to 
share power. It also appears difficult for some 
development systems to adjust readily to real 
shifts of power towards the periphery. Such 
shifts involve changes in patterns and responsi- 
bilities for decision making and resource alloca- 
tion. They also reveal differences in conceptual 
frameworks between those at the ‘top’ and 
those at the ‘bottom’. 


While some of the shortcomings of conven- 
tional evaluation approaches have been outlined 
above, it is not the intention of this article to 
suggest that participatory evaluation approaches 
are somehow a ‘panacea’ for all such shortcom- 
ings. Participatory evaluation approaches have 
their own characteristics and strengths, but they 
too at present have shortcomings. Both are con- 
sidered in the following sections. They are 
examined in the context of four specific expe- 
riences of training participants for participatory 
evaluation in countries in the Far East and 
Pacific Region between 1983-1986. 


Characteristics of the training experi- 
ences 


All. training experiences were related to the 
rural areas of developing countries. The follow- 
ing matrix provides basic information as to 
general location and characteristics of the train- 


ing : 


Location Length/ Type Participants Range/Educational 
of Training Technical Levels 
A. South Asia Two-day intensive _ District level Primary 
training preceding government staff Secondary 
six-week evaluation from health, edu- Tertiary 
of inter-sectoral cation, agriculture Postgraduate 


community health community develop- Technical 
development pilot ment and forestry 
project departments; local 
government officials; 
community health 
workers 


B. Northern Pacific Intensive training 


Region during 10-day veterinaries, Secondary 
conferertce of 27 CD staff, religious Tertiary 
non-governmental staff, community Postgraduate 
community based health workers (one- Technical 
and social quarter of participants) 
development 
programmes 


Physicians, nurses, Primary 


C. North Asia One-day exercise Government health Primary 


during one-month staff from Secondary 
training to national, provin- Tertiary 
establish pilot cial and district Postgraduate 
primary care level; government Technical 
project officials 
D. Central On the job Project staff Majority had sixth 

America training during from Peasant grade primary 
three-month Women’s education; one 
evaluation of Federation and illiterate 
non-governmental community health 
health and social workers 


development project 
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previous experiences in participatory evalua- 
tion. The modules (each not exceeding two 
pages) were designed for one to two hours’ 


The individual country settings vary consi- 
derably in terms of location, culture, language, 
political system and population. However, an 


analysis of these different training processes 
generates findings which may be of relevance to 
existing and intending practitioners of partici- 
patory evaluation approaches. 


Preparations for training 


The most basic preparation for training in all 
cases was to ensure that project participants and 
interested groups (government departments, 
agencies and institutions ) had a common under- 
standing of the meaning and purpose of partici- 
patory evaluation. In the North Pacific country a 
set of training modules had been prepared by 
the evaluation facilitator with representatives 
from the community based health programmes 
concerned. These were based on study of the 
programmes (some in their actual contexts), 
discussions with staff and communities and 


sessions during the 10-day training workshop 
for 82 participants. They followed a sequence 
designed to provide a basic conceptual frame- 
work for practitioners. The modules began by 
posing several basic questions such as “What ts 
Participatory Evaluation” and “Who is it for?” 
They then continued through the basic evalua- 
tion process of planning, selecting and testing 
methods, data collection, analysis, reporting and 
use of findings. 


Participatory evaluation was presented as a 
circular process-with the findings and conclu- 
sions linked closely with plans for the future 
action of the on-going project. For most partici- 
pants this concept was novel. They tended to see . 
evaluation as a more linear process which starts 
with the arrival of external evaluators and ends 
when they depart taking with them the data 
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collected during the evaluation process. 


While participants in three of the cases were 
familiar with monitoring activities, the idea that 
they themselves were to participate in a major 
way in the evaluation process was novel. In the 
South Asian country district staff from various 
development sectors and local government offi- 
cials had been informed through a ‘top-down’ 
order that an evaluation was to take place. The 
time scale of the evaluation had already been 
agreed. Within this context preparations were 
made for a two-day training workshop, using 
selected and adapted modules. The brevity of 
the training for participatory evaluation was 
particularly offset by the greater homogeneity 
and smaller number (20) of the group than in 
case B (82). 


An important feature of training (however 
brief) was to enable participants to gain confi- 
dence in their own evaluation capabilities. Par- 
ticular emphasis was placed from the start on 
analysing their customary methods of monitor- 
ing and management. For example the range of 
record-keeping was examined. Selected analysis 
of the data collected revealed strengths and 
weaknesses in the monitoring system, relevance 
to programme objectives and assisted partici- 
pants in framing evaluation objectives and 
planning for the approaches and methods requi- 
red in their respective evaluations. 


The participants in case A came from various 
development sectors. This required extra time 
to be devoted to an explanation of the different 
sectoral functions and the terminologies of the 
various sectors. Time was however saved by 
participants not considering it necessary to 
devote too much time to discussing why to eval- 
uate. From their point of view they had received 
an order to do so and appeared content to carry 
it Out. 


In cases A to C participants were involved in 
designing the training curricula. This consti- 
tutes the first part of any participatory training 
process. Any externally designed curricula impo- 
sed on participants would not have enabled 
them to understand the rationale for training, 
why specific training elements were necessary, 
how the elements needed to be placed in 
sequence or how their own skills and abilities 
could be used and expanded. 


It is said there are two ways of learning to 
swim: one way is to learn gradually at the shal- 
low end of a pool. The other is to make a 
dramatic entry into the deep end. In the latter 
case, it is suggested that the art of swimming can 
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be more rapidly acquired. Without digressing 
into the water-logged flaws in this argument, it 
can be said that the subsequent performance of 
participants in case D indicated that they had 
indeed accomplished the main learning objec- 
tives over the three-month period. However, 
had a short training been provided before the 
evaluation perhaps the evaluation process would 
have been less arduous. In all cases there was an 
emphasis on using locally available and low-cost 
materials such as writing paper, poster paper, 
and writing instruments. In cases A and B such 
materials were not in short supply due to the 
entrepreneurial ventures of local merchants. In 
C and D such materials were both scarce and 
very expensive. They had to be carefully 
‘rationed’. 


Planning evaluation and identifying 
objectives 


Some confusion arose in cases A, B and D over 
which were project objectives and which were 
evaluation objectives. The links between the 
two were obvious, but there were subtle and 
important differences which participants needed 
to grasp. A useful exercise in assisting partici- 
pants to identify evaluation objectives involved 
posing questions. For example, in case A partic- 
ipants divided their project activities into dif- 
ferent categories such as health, agriculture, 
education, community development, commu- 
nity involvement and ‘research and develop- 
ment’ surveys. Within these categories partici- 
pants then identified the questions which needed 
to be addressed during the evaluation. 


Participants did not initially find it easy to 
differentiate between questions which would 
provide data on process, e.g., how many patients 
are being seen per week by individual village- 
based community health workers, and those 
which aimed to reveal what impact the project 
was making, such as what changes had there 
been in the patterns of morbidity and mortality 
during the past two years. Groups of parti- 
cipants were elected to assume special responsi- 
bility for identifying the approaches and methods 
by which the questions could be answered: for 
example, a questionnaire was designed to evalu- 
ate the effectiveness of the original training 
received by programme staff who trained com- 
munity health workers. Meetings and field trips 
were planned and arranged to enable commu- 
nity representatives to participate in evaluating 
project progress in their own areas. 


One general problem encountered by partici- 


pants in all cases was understanding the pur- . 


poses and types of indicators of progress. In case 
B indicators were likened to milestones’ which 
could inform participants of progress achieved 
during a journey. The analogy, presented visu- 
ally, worked well. During training it was nece- 
ssary to translate into clear, and where possible 
visual terms, many of the often complex con- 
cepts and processes of evaluation. This work of 
‘de-mystification’ is fundamental if the learning 
process is to be successful. There are at present 
relatively few training materials of this sort 
available. In this training and planning approach 
a project is deliberately ‘fragmented’ in order 
that participants can perceive these fragments 
and how they fit—or should fit—together in 
order to achieve certain project objectives. This 
stage of training often reveals original project 
design faults. As this can be a sensitive finding 
an atmosphere of rapport and trust must 
already have been established. During training 
the actual evaluation plan is constructed and a 
‘core group’ of participants assume responsibil- 
ity for carrying it out within a specific time 
frame. A chalkboard and/or whiteboard are use- 
ful for planning purposes as adjustments can 
more easily be made than on paper. It is impor- 
tant to have a centrally located and easily availa- 
ble evaluation plan. A smaller board is useful for 
daily or weekly monitoring of evaluation acti- 
vities. 


Part of planning for participatory evaluation 
involves thinking about how community repre- 
sentatives and members will be involved in the 
evaluation process. In case D only community 
level participants were involved with the facilit- 
ator. The core evaluation team prepared a two- 
page information sheet outlining the purpose, 
process and sequence of their intending evalua- 
tion. This was widely distributed in the project 
area and feedback from village level was used to 
guide the subsequent evaluation process. 


In case A community representatives did not 
attend the training due to its brevity and 
urgency. However, they were involved in data 
collection, analysis and discussion of findings. In 
case B community representatives constituted 
abcut 25 per cent of all participants. In case C no 
community participants were involved in the 
evaluation exercise to establish the pilot project 
as it took place during a national level training 
course. 


Learning to use the tools of evaluation 


During training the selection and preparation 
of evaluation methods have to be made bearing 
in mind the objectives of the evaluation, the 


experience and capabilities of participants and a 
combination of locally determined factors such 
as geographic conditions, distances involved, 
materials and reproducing facilities available. 
The aim is one of producing and using the best 
range of methods appropriate to the circum- 
stances. 


Participants are helped to examine their 
existing methods of management and monitor- 
ing in order to adapt what is useful for the 
evaluation. They may be familiar with how sur- 
veys and questionnaires are constructed and 
pretested. Participants may only be familiar 
with a limited range of evaluation methods. 
They will need to expand and adapt them. For 
example, in case D due to time constraints a 
group questionnaire method was adopted. Mem- 
bers of the core evaluation team held village 
meetings. Using a school blackboard, each fam- 
ily was given a code number and then the whole 
group provided data which was ranged on the - 
prepared blackboard matrix. Village members 
then participated in analysing and discussing 
the results. 


Interview techniques are difficult for partici- 
pants to learn in a short time. By drawing out 
some main guidelines certain skills can be 
learnt. However, there is usually insufficient 
time to practice them. The actual field-testing 
(1.e. piloting the methodology) is not the best 
time for training interviewers. In practice unfor- 
tunately circumstances may make it very diffi- 
cult to do it at any other time. The only safe- 


-guards appear to be in selecting interviewers 


very carefully and in identifying individuals who 
have the range of interpersonal skills necessary 
for effective interviewing. Also, constant feed- 
back, while interviewing is being carried out, 
can help improve practice while it develops. 


In all cases participants generally experienced 
difficulty in attending to the meticulous detail 
on which effective evaluation methodology 
must be based. This occurred irrespective of the 
educational or technical levels or the practical 
experiences of participants. Even where partic- 
ipants were meticulous in carrying out what 
they saw as ‘normal duties’, they had initial 
difficulty in translating this characteristic to the 
rigours of evaluation methodology. Perhaps this 
was partly because evaluation had been regarded 
as ‘somebody else’s responsibility’, e.g.,an out- 
side evaluator’s. Also until participants adopt 
responsibility for the evaluation it may appear 
as an extra burden to practitioners who already 
carry a heavy workload. 


But once the adoption process has taken place 
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the momentum of the evaluation produces a 
speed and rhythm of its own. Participants are 
borne along on alternating waves of enthusiasm 
and fatigue as the demanding process evolves. It 
is here that the role of an evaluation facilitator is 
of particular importance. During periods of 
fatigue and even alienation, the facilitator non- 
directively provides support and guidance to 
participants. During training it is therefore 
important that participants are alerted to the 
possible rhythms of the evaluation process. In 
cases A and B participants benefited from learn- 
ing through a case-study and slide presentation 
of the experiences of others who had carried out 
participatory evaluation in countries with cir- 
cumstances similar to their own. 


Training for data analysis 


This part of the training focused on two specific 
types of data used in participatory evaluation. 
The first was data which already existed such as 
project records, reports and survey results. The 
second was new data which was collected during 
the evaluation period such as that resulting 
from interviews, questionnaires, surveys, meet- 
ings and case studies. 


The training for data analysis began by assist- 
ing participants to focus on their normal man- 
agement and monitoring system. They did this 

‘by analysing samples of data. In case A content 
analysis was carried out on all available health 
data (health centre, hospital, mobile teams and 
community health worker records) for a single 
village over a specific period. Thjs revealed, 
among other findings, the need to adjust the 
record-keeping system as some records were 
using different age and disease and gender clas- 
sifications, which prevented certain collations. 
Changes in one part of an information system 
usually necessitate changes in the whole system. 
Within a bureaucratic system this takes time. In 
all cases (A to D) the feedback loops in the 
management information system were negligi- 
ble. Information generally proceeded upwards 
and rarely back to its original generators. Those 
who collected and collated the information had 
not been encouraged to analyse it. Analysis was 
regarded as the function of a higher official. 


From analyses arise conclusions. From con- 
clusions decisions are made. By clinging to nar- 
row definitions of whois eligible to analyse data, 
the pool of decision-makers will continue to be 
narrow. Experience is beginning to indicate the 
value of increasing the analytical responsibili- 
ties and capacities of a wider range of people at 
different levels of a management information 
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system. Such a system is less prone to generate 
invalid data, as those involved understand the 
purpose and implications of the data gathered. 


In case C survey data had been collected prior 
to the course. The responsibility for all such data 
analysis had been entrusted to a newly formed 
national level computerized Statistics Unit. Due 
to work pressure, however, the unit had not 
been able to analyse the district survey data. The 
training course needed that data as one base for 
participatory curriculum development to ensure 
that the course was tailored to the needs of the 
district as revealed in the survey. It became 


possible for the Statistics Unit to release the 


data for participants to carry out primary analy- 
sis. This took place during a two-hour period 
with 30 ‘participants and a course facilitator 
using two blackboards. Different groups col- 
lated piles of survey data sheets and the result- 


ing data was recorded for all groups on the 
blackboard. 


This analytical and visual group activity was 
able to provide participants (some of whom had 
taken part in the survey) with both a simple 
participatory mechanism for rapid data analysis 
and in this case the survey results they needed to 
design and develop their own training course. 


Training in analysis also needs to extend to 
community based workers who can use various 
methods such as tally sheets, not just to collect 
and collate data, but also to analyse its impltca- 
tions. For community level participants numer- 
acy presents a problem in basic quantitative 
analysis. In case D the availability and interpre- 
tation of quantitative data was strongly related 
to the educational grades and numeracy of the 
peasant women concerned. The evaluation pro- 
cess had to be tailored to their capabilities and. 
needs. In case A the training process involved a 
much wider range of participants—from physi- 
clans to peasants. Bearing in mind the basic 
principle that process and outcomes had to be 
comprehensible to all, it was not always easy to 
find effective solutions which were fully appre- 
ciated by those with primary education grades 
only and which did not at the same time slightly 
frustrate those with postgraduate levels of edu- — 
cation and technical training. 


It is necessary to further develop the range of | 
quantitative evaluation methods used by partic- 
ipants who have few grades of primary educa- 
tion or who are not literate. The basis of such 
methods must be an understanding of the way 
in which peopie at community level normally 
count and measure things. For example, a 
farmer in the Philippines places a grain of corn 


in a tin for every rod of tobacco leaves that he 
plans to sell to the tobacco merchant. Then he 
counts the grains of corn so that he knows how 
many rods he has and how much money he 
should receive. 


Reporting and using evaluation findings 


For an evaluation report to be ‘action oriented’ it 
must be in a form easy to understand and use. 
During training the facilitator built up partici- 
pants confidence in their own capability to pro- 
duce such a report. Many participants had never 
seen an evaluation report. Those who had, 
found them lengthy, complex and partly incomp- 
rehensible. Such reports are usually cornpiled 
for a limited audience. Experience indicates that 
once the basic principles of effective reporting 
are grasped there are a whole range of presenta- 
tions that can be produced by participants. In 
case A the evaluation process culminated in a 
seven-day workshop where findings were shared 
and discussed and compiled into a 127-page 
report. Community leaders then suggested that 
if project staff could prepare a 2-3 pages report, 
this could be shared and discussed at the annual 
general community meeting. In case B, partici- 
pants developed a range of posters and visual 
presentations suitable for sharing with com- 
munity level audiences. These included the use 
of shapes cut from leaves of various colours to 
make a simple poster representing local health 
statistics. Although such materials were not 
long lasting they were easily available and cost 
nothing. In case D peasant women participants 
themselves wrote the greater part of their 95 
pages evaluation report, which contained a few 
tables. This was then distributed to key project 
personnel in the province. 


During training participants also had to be 
alerted to the phenomena of ‘embarrassing’ or 
‘unwelcome’ findings. Such findings may be the 
reason why some evaluation reports suffer the 
fate of ‘file and forget’, in other words the 
results are not shared. 


The extent to which findings are in fact used 
is influenced by a variety of factors. The will to 
implement the findings may exist but the 
resources to do so may not. On the other hand 
additional action may result. In case D the pro- 
ject was unable to obtain the further technical 
training desired within the country. By obtain- 
ing a modest grant they were able to travel 
across the national frontiers to obtain that train- 
ing in a project similar to theirs ina neighbour- 
ing country. Prior to the evaluation they had 
hardly been out of their own province. 


The facilitator’s multifaceted role 


It has been said that the presence of an expe- 
rienced and sensitive ‘outsider’ can encourage a 
group to formulate and articulate its thoughts 
more systematically and objectively and enrich 
discussions by bringing in other experiences, 
perceptions, perspectives and dimensions. There 
can be areas which local people either forget to 
look at, or do not want to look at. An outsider 
can play an important role by asking the right 
kind of questions and providing useful insights 
into dealing with dilemmas and uncertainties. 
Evaluation methodology forms part of many 
vocational and university courses, but is often 
presented only as part of a management pro- 
cess. While the monitoring is stressed, the wider 
and deeper issues relating to impact evaluation 
(1.e., relating to outcomes) may be passed over 
rapidly. Such issues are deemed to belong more 
to the realm of the ‘expert evaluator’. This 
undercuts the confidence of a wider range of 
people to assume the role of ‘evaluation facilita- 
tor. In addition, many teaching/learning styles 
In institutions and universities continue to be 
blatantly non-participatory. In such situations’ 
the learner receives little opportunity to develop 
attitudes and skills designed to make learning a 
mutual process between teacher and learner. 
The ‘teacher’ in a participatory evaluation pro- 
cess is both a ‘learner’ anda ‘researcher’. In such 
a process the task of the researcher becomes not 
to produce knowledge but to facilitate the con- 
struction of knowledge by the community itself. 


From the experiences briefly described in this 
article somé general guidelines emerge relating 
to the complex and demanding role of a partici- 
patory evaluation facilitator, in both the train- 
ing and evaluation processes. 


i. A relationship of trust and confidence 
must be rapidly established between facil- 
itator, project staff and participants. 


ii. Where possible the facilitator should live 
directly with project staff and participants 
thus increasing the range of mutual learn- 
ing opportunities and shared experiences. 


iii. The role that the facilitator expects to play 
must be clarified and agreed early in the 
relationship. 


iv. Negotiation is also necessary between all 
interested parties (government, agencies, 
project staff and participants) regarding 
evaluation objectives and methodology. 


v. The facilitator must be familiar with the 
rationale and basic approaches of partici- 
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patory learning and teaching. 


vi. A non-directive leadership style is the 
most appropriate, with the facilitator as 
suming overall responsibility within the 
context of shared leadership. 


vii. The facilitator must adapt the leadership 
style to provide more support where 
needed and less where existing competen- 
cies are strong. Sensitivity in changes in 
the evaluation process is essential. 


viii. Abilities are needed to diffuse and resolve 
situations of tension and also possible dif- 
ferences in viewpoints during the arduous 
évaluation process. 


ix. Abilities are also needed to withstand pos- 
sible degrees of animosity during the eva- 
luation process as participants more fam1- 
liar with ad hoc procedures may consider 
the facilitator unnecessarily meticulous as 
far as attention to detail is concerned. 


x. Only as the facilitator becomes sufficiently 
familiar with the context and operational 
procedures of the project can the final 
choice of appropriate evaluation methods 
be made. — 


xi. The real challenge lies not in getting the 
people to fit the methods, but in getting 
the methods to fit the people. 


Concluding comment 


A participatory evaluation approach is not only 
an evaluative but also an educational approach. 
In addition it increases the management capaci- 
ties of project staff and participants. In this 
steadily evolving approach there is a need for 
increased contact and cooperation betwéen its 
practitioners. From the brief analysis of the four 
cases of training for participatory evaluation on 
which this paper is based, common principles, 
practices and constraints have been drawn out. 
It is actual practice that must guide the further 
refinement of the approach. The evolution of an 
ever widening range of flexible methodology is 
needed in line with the needs of different cultu- 
ral and political contexts, educational levels and 
the characteristics of the various development 
sectors. 


Finally, perhaps the most fundamental con- 
straint to the more rapid adoption of participa- 
tory evaluation remains at a deep conceptual 
level. A ‘conceptual framework’ is like a system 
of values and attitudes which result in a specific 
view of reality. Conventional evaluation appro- 


80 


aches have been based largely on a narrow set of 
western and elite-dominated conceptual frame- 
works. For this reason such approaches have 
given scant recognition to the validity of a wider 
range of frameworks including those found at 
community level. Or, to put it another way, are 
the powerless and poor to continue to be judged 
by others. using sets of assumptions and value 
judgements constructed very largely without 
their participation ? 
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